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Abstract 

Developmental  feedback  programs  provide  an  opportunity  to  improve  leadership 
practices  by  giving  leaders  feedback  about  their  performance  from  a  number  of  sources. 
This  thesis  expanded  on  Capt  Doug  Patton’s  pilot  study  (2002)  that  developed  and 
validated  an  upward  feedback  instrument  (UFI)  for  Aeronautical  Systems  Center  (ASC) 
and  Air  Force  Security  Assistance  Center  (AFSAC)  supervisors.  Specifically,  the 
purpose  of  this  thesis  was  to  develop  a  web-based  developmental  feedback  collection, 
reporting,  and  analysis  tool-set.  The  collection  and  reporting  instrument  was  based  on 
Patton’s  instrument  (2002)  and  updated  under  an  on-going  study  by  Capt  Abby  White 
(2003).  The  analysis  tools  were  loosely  based  on  the  exercises  contained  in  the 
Leadership  Challenge  Workbook  (Kouzes  and  Posner,  2001)  as  well  as  recommendations 
from  other  leadership  scholars  (e.g.,  Yukl,  2002). 

A  structured  approach  to  the  information  systems  life  cycle  was  used  to  design  the 
web-based  developmental  feedback  system.  The  result  of  the  life  cycle  was  a  system  that 
performed  three  distinctive  functions:  1)  on-line  collection  instruments  for  self  and 
observer  assessments,  2)  automatic  feedback  data  entry  and  reports  for  each  leader  and 
across  leaders  for  the  researcher,  and  3)  on-line  workbook  to  aid  leader  self-analysis  and 
development  of  action  plans.  The  perceived  utility  of  the  feedback  provided  by  the 
program  and  usability  of  the  system  were  assessed.  Results  from  the  assessments  were 
used  to  recommend  improvements  to  the  program. 


xi 
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DEVELOPING  A  WEB-BASED  DEVELOPMENTAL  FEEDBACK  PROGRAM 

I.  Introduction 

Background 

Developmental  feedback  programs  provide  an  opportunity  to  improve  leadership 
practices  by  giving  leaders  feedback  about  their  perfonnance  from  a  number  of  sources. 
This  thesis  expanded  on  Patton’s  pilot  study  (2002)  that  developed  and  validated  an 
upward  feedback  instrument  (UFI)  for  Aeronautical  Systems  Center  (ASC)  and  Air  Force 
Security  Assistance  Center  (AFSAC)  supervisors.  Specifically,  the  purpose  of  this  thesis 
was  to  develop  a  web-based  developmental  feedback  collection,  reporting,  and  analysis 
tool-set.  The  collection  and  reporting  instrument  was  based  on  Patton’s  instrument 
(2002)  and  updated  under  an  on-going  study  (White,  2003).  The  analysis  tools  were 
loosely  based  on  the  exercises  contained  in  the  Leadership  Challenge  Workbook  (Kouzes 
and  Posner,  2001)  as  well  as  recommendations  from  other  leadership  scholars  (e.g.,  Yukl, 
2002). 

The  upward  feedback  instrument  developed  by  Patton  (2002)  was  designed  to 
measure  six  different  leadership  constructs.  Five  constructs  were  modeled  after  the 
practices  presented  in  The  Leadership  Challenge  by  Kouzes  and  Posner  (1995).  The  five 
practices  were  challenge  the  process,  inspire  a  shared  vision,  model  the  way,  enable 
others  to  act,  and  encourage  the  heart.  Kouzes  and  Posner  (1995)  defined  two 
commitments  for  each  practice  that  described,  in  further  detail,  actions  people  should  take 
to  develop  their  leadership  behaviors.  Posner  and  Kouzes  (1988)  developed  the 
Leadership  Practices  Inventory  (LPI)  to  measure  each  of  the  five  leadership  practices. 
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Patton’s  sixth  construct,  have  fun,  was  developed  at  the  request  of  ASC  and  AFSAC,  and 
reflected  the  extent  to  which  supervisors  engaged  in  behaviors  that  were  designed  to 
relieve  stress  and  tension  in  the  workplace.  In  addition  to  the  sixth  leadership  practice, 
have  fun,  Patton’s  UFI  differed  from  the  LPI  in  that  it  measured  leader  behavior  at  the 
lower  commitment  level. 

The  leadership  commitments  inventory  (LCI)  was  the  instrument  designed  for  this 
study’s  web-based  developmental  feedback  program.  White  (2003)  modeled  and 
developed  the  LCI  using  the  research  conducted  by  Kouzes  and  Posner  (1995,  2002)  and 
Patton  (2002).  It  maintained  the  original  five  practices  from  Kouzes  and  Posner’s  LPI, 
and  the  sixth  practice  from  Patton’s  UFI,  now  called  “Enjoy  the  workplace.” 

The  LCI  updated  over  half  of  Patton’s  items  in  order  to  more  distinctively 
measure  the  twelve  commitment  constructs  and  to  reflect  changes  made  by  Kouzes  and 
Posner  (2002).  All  items  were  updated  to  facilitate  360-degree  observation. 

Research  Objectives  and  Scope 

Feedback  programs  only  provide  value  to  the  extent  people  use  them.  To  change 
or  improve  their  leadership  behaviors,  leaders  must  move  through  four  stages  of 
feedback.  These  stages  are:  (a)  feedback  is  perceived,  (b)  feedback  is  accepted  as 
accurate,  (c)  feedback  is  found  useful,  and  (d)  actual  behavior  is  changed  (Brett  & 
Atwater,  2001).  An  effective  developmental  feedback  program  should  provide  the  tools 
necessary  for  the  leader  to  move  through  the  four  stages.  An  effective  tool  should  also  be 
fairly  efficient.  The  original  instrument  was  a  paper  and  pencil  questionnaire.  Data 
needed  to  be  entered  manually  and  the  researchers  generated  reports  through  a  labor- 
intensive  process  integrating  data  across  multiple  spreadsheets.  Reports  were  provided  in 
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paper  format.  The  commercially  available  workbooks  were  tailored  with  additional 
instructions  for  the  have  fun  construct,  as  well  as  idiosyncratic  differences  between  the 
commercial  and  developed  instruments.  No  facilities  were  provided  to  allow  supervisors 
to  track  their  performance  over  time. 

The  overall  objective  of  this  study  was  to  develop  a  web-based  system  for  the 
developmental  feedback  programs  of  ASC,  AFSAC,  and  Air  Force  Materiel  Command 
(AFMC)  that  could  move  leaders  through  the  feedback  process  while  automating  data 
collection,  reports,  and  statistical  analysis.  There  are  many  advantages  to  web-based 
systems.  Some  include  high  rate  of  response,  short  time  frame  for  the  collection  of 
responses,  and  time  and  cost  savings  (Clarkson,  1999;  Coomber,  1997;  Kimball,  1998; 
Mertler,  2002;  Sheehan  &  Hoy,  1999;  Sills  &  Song,  2002;  Slevin,  1997;  Tierney,  2000; 
Virtual  Survey  Limited,  1996;  Watt,  1997).  Furthermore,  they  are  an  excellent  resource 
for  obtaining  and  processing  large  amounts  of  data  (Mertler,  2002).  This  study  aimed  to 
develop  a  web-based  system  with  three  distinctive  functions:  1)  on-line  collection 
instruments  for  self  and  observer  assessment,  2)  automatic  statistical  analysis  and 
generation  of  reports  for  each  leader  and  across  leaders  for  the  researcher,  and  3)  on-line 
workbook  to  aid  leader  self-analysis  and  development  of  action  plans.  These  functions 
were  specifically  designed  to  move  a  leader  through  the  feedback  process  to  improve 
their  leadership  behaviors. 


Research  and  Investigative  Questions 

To  accomplish  these  objectives,  it  was  necessary  to  answer  specific  research  and 
investigative  questions.  The  overall  research  question  was  “How  can  a  web-based 
developmental  feedback  program  provide  the  necessary  tools  for  leaders  to  move  through 
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the  four  stages  of  feedback  and  improve  their  leadership  behaviors?”  The  investigative 
questions  were  as  follows. 

1 .  What  system  capabilities  are  dictated  by  developmental  feedback  requirements 

that  will  allow  the  leader  to  easily  move  through  the  feedback  process  to  behavior 

change? 

2.  How  do  the  requirements  affect  system  architecture? 

3.  How  does  the  system  architecture  affect  the  actual  system  design? 

4.  How  do  you  translate  the  architectural  and  physical  designs  into  a  working 
information  system? 

5.  How  do  you  assess  the  perceived  utility  of  the  feedback  and  usability  of  the 
information  system? 

Importance  of  this  Research 

This  study  retrieved  and  consolidated  the  theoretical  requirements  for  an  effective 
developmental  feedback  program,  providing  a  baseline  program  for  organizations.  This 
program  was  designed  to  provide  leaders  with  an  accurate  portrayal  of  their  perfonnance, 
allowing  them  to  make  meaningful  behavioral  changes  that  will  improve  their  leadership 
and  the  organization.  This  study  is  also  beneficial  to  researchers  in  that  it  created  a 
platform  for  them  to  launch  developmental  feedback  studies.  They  now  have  a  tool  that 
will  provide  automatic  data  input  and  statistical  analysis,  saving  time  and  money. 
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II.  Literature  Review 

This  literature  review  contains  four  sections.  The  first  section  focuses  on  what 
makes  an  effective  developmental  feedback  program  and  how  such  a  program  can 
provide  accurate  and  useful  feedback.  The  second  section  highlights  the  theory  behind 
and  development  of  the  Leadership  Practices  Inventory  (LPI),  Upward  Feedback 
Instrument  (UFI),  Leadership  Commitments  Inventory  (LCI),  and  Stakeholder 
Assessment  feedback  instruments.  It  also  addresses  the  reasons  why  the  LCI  was 
developed  and  used  in  place  of  the  LPI  and  UFI.  In  the  third  section,  the  advantages  of 
web-based  surveys  are  discussed.  Finally,  the  fourth  section  focuses  on  information 
systems  development. 


Developmental  Feedback 

Developmental  feedback  (also  referred  to  as  360-degree  feedback,  multi-rater 
feedback,  or  multi-source  feedback)  can  be  provided  by  a  number  of  sources.  Sources 
can  include  a  leader’s  boss,  peers,  subordinates,  team  members,  or  customers. 
Developmental  feedback  programs  focus  on  providing  positive  change  for  the  leader  and 
the  organization  (Bracken,  1994).  An  effective  program  will  highlight  leadership 
development  areas  for  the  leader,  encourage  the  leader  to  set  goals  based  on  desired 
improvement  in  those  areas,  and  ultimately  lead  to  behavior  change  (London  &  Smither, 
1995).  To  implement  a  successful  program,  top  management  should  introduce  the 
program  to  let  leaders  know  that  developmental  feedback  is  important  to  the 
organization.  Leaders  must  also  be  given  the  discretion  to  seek  feedback,  develop  action 
plans,  and  change  behavior  based  on  the  feedback  (Alimo-Metcalfe,  1998). 
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Ilgen,  Fisher,  and  Taylor  (1979)  posited  that  an  individual’s  processing  of 
feedback  can  be  broken  into  four  stages:  (a)  perception  of  feedback,  (b)  acceptance  of 
feedback,  (c)  desire  to  respond  to  feedback,  and  (d)  the  intended  response.  Perception  of 
feedback  is  concerned  with  how  accurately  the  recipient  perceives  the  feedback  from  any 
given  source.  These  perceptions  are  affected  by  the  interval  between  the  behavior  and 
the  feedback  received  (timing),  whether  or  not  the  feedback  was  positive  or  negative 
(sign),  and  how  often  feedback  is  provided  (frequency).  Feedback  acceptance  refers  to 
the  recipient’s  belief  that  the  feedback  is  an  accurate  portrayal  of  his  or  her  behavior  or 
performance.  Here  feedback  is  most  likely  accepted  as  accurate  if  the  source  was  reliable 
and  credible  and  if  the  feedback  was  positive.  The  desire  to  respond  to  the  feedback  is 
linked  to  the  power  of  the  feedback  source.  Power  refers  to  the  extent  to  which  the 
recipient  believes  the  source  influences  the  contingency  between  the  recipient’s  behavior 
and  his  or  her  receipt  of  valued  outcomes.  In  other  words,  the  more  the  source  controls 
valued  outcomes,  the  more  likely  the  recipient  will  respond  to  the  feedback. 

The  intended  response  is  motivated  by  the  feedback  received,  which  plays  an 
essential  role  in  the  goal  setting  process.  But  feedback  alone  will  not  produce  behavior 
change  (Locke  &  Latham,  1990,  2002).  Therefore,  feedback  must  allow  the  recipient  to 
set  specific,  difficult  goals.  As  a  result,  that  person  will  be  more  likely  to  produce  the 
intended  response.  However,  the  feedback  process  does  have  a  constraint.  If  the 
recipient  does  not  have  the  discretion  (i.e.,  support  of  supervisor  or  organization)  to 
change  behavior,  the  feedback  provided  cannot  be  acted  upon. 

Brett  and  Atwater  (2001)  applied  the  feedback  model  and  assessed  the  four  stages 
in  understanding  how  developmental  feedback  results  in  behavior  change.  The  stages 
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were  adapted  to  their  study  and  were:  (a)  feedback  is  perceived,  (b)  feedback  is  accepted 
as  accurate,  (c)  feedback  is  found  useful,  (d)  actual  behavior  is  changed.  Effective 
developmental  feedback  programs  will  provide  the  tools  necessary  for  the  leader  to  move 
through  the  four  stages. 

After  sifting  through  the  literature  on  developmental  feedback  it  became  apparent 
that  effective  developmental  feedback  programs  provided  three  functions:  (a)  feedback 
collecting,  (b)  results  reporting,  and  (c)  action  planning.  These  functions,  if  properly 
designed  and  implemented,  work  together  to  help  the  leader  move  through  the 
aforementioned  stages  of  feedback. 

Organizations  should  tailor  these  functions  to  meet  their  needs.  The  following 
paragraphs  highlight  each  function.  The  features  of  each  function  are  ideal  to  implement, 
however  organizations  may  not  be  able  to  institute  all  of  the  features  due  to  resource 
constraints  and  practicality. 

Feedback  Collecting 

To  collect  feedback,  an  organization  needs  a  developmental  feedback  instrument 
and  raters.  There  are  numerous  available  instruments  but  organizations  should  work  hard 
to  choose  or  develop  an  instrument  that  meets  their  needs.  First  and  foremost,  the 
instrument  should  be  reliable  and  valid  (Bracken,  1994;  Fletcher,  Baldry,  & 
Cunningham-Snell,  1998;  Morical,  1999;  Vinson,  1996)  and  should  have  been  developed 
based  on  statistical  methods  (Vinson,  1996).  To  ensure  complete  and  purposeful 
feedback,  the  category  structure  of  the  instrument  should  be  developed  using  factor 
analysis  (Bracken,  1994;  Hazucha,  Hezlett,  &  Schneider,  1993).  This  breaks  the  items 
into  specific  areas  of  focus  and  provides  the  leader  more  specific  direction  for 
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individualized  development  planning.  The  areas  of  focus  should  highlight  behaviors  (not 
traits)  that  are  important  to  the  organization  as  well  as  the  organization’s  values  (Bracken, 
1994;  London  &  Smither,  1995;  Morical,  1999).  This  lets  the  leader  know  which 
leadership  behaviors  are  critical  for  success.  All  these  features  ensure  the  leader  being 
assessed  gets  accurate  information.  The  instrument  can  be  administered  as  a  paper-and- 
pencil  survey  or  electronically.  The  advantages  of  an  electronic  (or  web-based)  survey 
will  be  discussed  later. 

While  choosing  an  instrument,  organizations  should  choose  one  that  allows  the 
leader  to  perform  a  self-assessment  (Church,  1995;  London  &  Smither,  1995).  Self- 
assessment  is  important  for  a  couple  of  reasons.  It  allows  the  leader  to  view  the  relevant 
leadership  behaviors  and  gain  an  initial  understanding  of  some  of  the  areas  they  might 
need  to  improve.  It  also  provides  information  for  the  leader  to  compare  his  or  her  ratings 
with  the  ratings  of  others,  thereby  gaining  a  better  self-understanding  (London  & 

Smither,  1995).  After  the  self-assessment,  the  leader  should  request  feedback  from  other 
sources.  Organizations  determine  who  will  provide  the  feedback  and  should  choose  an 
instrument  that  allows  for  multiple  sources  of  feedback  (i.e.,  boss,  peers  or  coworkers, 
subordinates). 

These  raters  are  obviously  an  extremely  important  part  of  the  developmental 
feedback  process.  Therefore,  choosing  the  correct  raters  should  be  at  the  forefront  of  any 
leader’s  mind  (Alimo-Metcalfe,  1998;  Bracken,  1994).  London  and  Smither  (1995) 
identified  certain  rater  characteristics  that  affect  how  much  the  leader  will  pay  attention 
to  the  feedback.  The  feedback  source  must  be  reliable  and  credible  and  must  have  the 
power  to  affect  leader  response.  If  the  source  is  reliable  and  credible,  the  leader  will 
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more  likely  view  the  feedback  as  accurate.  Ilgen  et  al.  (1979)  predicted  that  a  feedback 
source  that  controls  valued  outcomes  (i.e.,  has  power)  will  most  likely  cause  a  recipient 
to  respond  to  the  feedback  and  provide  the  intended  response.  In  the  case  of  multi-source 
feedback,  the  leader’s  boss  would  seem  to  have  the  most  power  because  the  boss  controls 
many  valued  outcomes  such  as  pay,  benefits,  time  off,  and  promotions.  An  improvement 
seen  by  the  boss  may  have  an  effect  on  the  valued  outcomes.  It  would  seem  that  other 
sources  might  have  less  power  because  of  their  inability  to  control  valued  outcomes. 
However,  Brett  and  Atwater  (2001)  posited  that  direct  reports’  might  also  be  perceived  as 
the  most  relevant  because  the  leadership  behaviors  highlighted  are  most  applicable  to  that 
group. 

There  must  also  be  enough  raters  to  provide  accurate  feedback.  Theoretically, 
more  raters  will  assure  more  accurate  feedback.  Greguras  and  Robie  (1998)  posited  that 
the  number  of  raters  should  be  increased  for  better  interrater  reliability.  London  and 
Smither  (1995)  found  that  a  larger  number  of  raters  also  ensured  the  leader  paid  more 
attention  to  the  feedback  (i.e.,  deemed  it  more  accurate).  According  to  London  and 
Smither  (1995),  three  to  five  raters  reduced  the  possibility  of  biased,  inaccurate 
information. 

One  pitfall  sometimes  found  in  developmental  feedback  programs  is  that  leaders 
will  pick  their  friends  to  assess  their  leadership  behaviors.  As  a  result,  the  leader  does  not 
necessarily  receive  the  most  accurate  infonnation.  Organizations  can  reduce  the 
possibility  of  this  bias  occurring  by  limiting  the  number  of  raters  a  person  can  have,  by 
requiring  leaders  to  have  their  list  reviewed  by  someone,  or  by  limiting  the  number  of 
assessments  a  particular  rater  can  perform  (Bracken,  1994).  They  can  also  achieve  less 
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bias  through  encouragement  (get  better  information)  and  assurance  (critical  infonnation 
will  not  be  used  administratively). 

Organizations  should  ensure  the  feedback  is  confidential  and  anonymous  (Alimo- 
Metcalfe,  1998;  Bracken,  1994;  London  &  Smither,  1995;  Vinson,  1996).  Even  when 
programs  are  used  for  developmental  purposes  it  is  wise  to  guarantee  confidentiality  by 
using  a  party  outside  the  organization  to  process  the  data  and  provide  the  results.  In  other 
words,  the  feedback  data  should  only  be  used  to  provide  feedback  reports  for  the  leader. 
Furthermore,  the  leader  should  be  the  only  person  to  have  access  to  the  reports. 
Confidentiality  and  anonymity  encourage  honest  participation  by  leaders  and  raters  alike, 
resulting  in  more  accurate  feedback  (Bracken,  1994).  Most  feel  that  to  ensure  anonymity 
at  least  three  raters  should  be  used  in  a  particular  feedback  category  (Bracken,  1994; 
London  &  Smither,  1995). 

Results  Reporting 

Once  the  raters  provide  the  feedback,  it  must  be  presented  for  the  leader  to  receive 
and  understand  it.  Feedback  reports  are  a  good  way  to  integrate  feedback  results  into  the 
developmental  feedback  program  (Church,  1995).  These  reports  should  be  the  starting 
point  of  the  development  process,  allowing  leaders  to  review  their  results  and  prepare  a 
development  plan  (Alimo-Metcalfe,  1998).  The  results  should  be  clear,  specific,  and 
purposeful  in  nature  (Morical,  1999). 

A  complete  and  understandable  report  format  is  key  to  understanding  the 
feedback  and  can  affect  the  perceived  accuracy  of  the  results  (London  &  Smither,  1995). 
A  report  using  a  graphical  representation  of  the  results  allows  leaders  to  make  immediate 
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interpretations  of  the  feedback  and  makes  it  easier  for  them  to  translate  the  results  into 
action  (Morical,  1999). 

Comparison  reports  are  helpful  for  leaders  to  relate  others’  feedback  with  their 
own  self-assessment  (Church,  1995).  These  comparisons  allow  leaders  to  detennine 
discrepancies  between  their  perceptions  and  the  perceptions  of  others.  The  discrepancies 
provide  natural  targets  for  goal  accomplishment  in  order  to  restore  perceptions  of  self¬ 
competence. 

Ratings  of  the  instrument  items  or  categories  should  be  presented  with  some  index 
of  variability  (e.g.,  standard  deviation  or  range)  to  show  the  leader  the  degree  to  which 
the  raters  agreed  or  disagreed  (Gregarus  &  Robie,  1998;  London  &  Smither,  1995).  Once 
again,  in  order  to  ensure  anonymity  of  the  raters,  feedback  results  for  a  specific  category 
should  not  be  presented  until  three  raters  have  provided  feedback  (Bracken,  1994). 
However,  for  certain  categories  of  raters  (e.g.,  the  boss)  the  rule  of  three  does  not  apply. 
The  boss  is  accustomed  to  providing  feedback  and  likely  does  not  require  anonymity  to 
provide  an  honest  assessment. 

Action  Planning 

Feedback  alone  does  not  lead  to  skill  development  and  perfonnance 
improvement.  As  stated  before,  the  feedback  must  allow  the  leader  to  set  specific, 
difficult  goals  before  behavior  change  can  take  place  (Locke  &  Latham,  1990,  2002). 
Developmental  feedback  programs  should  have  some  sort  of  action  planning  mechanism 
to  allow  for  goal  setting.  In  this  case,  goal  setting  does  not  necessarily  have  to  be 
voluntary  but  should  occur  as  a  result  of  the  program  design  (London  &  Smither,  1995). 
Action  planning  or  follow-up  is  necessary  for  a  successful  program. 


12 


Once  leaders  review  the  results  in  the  feedback  report,  they  are  ready  for  skill 
development  or  improvement.  To  do  this  they  must  develop  some  type  of  action  plan. 
Action  planning  can  be  enhanced  through  one  of  three  ways:  a  one-on-one  facilitated 
development  session,  a  development  workshop,  or  a  development  workbook  (Bracken, 
1994). 

An  effective  way  to  help  leaders  develop  action  plans  is  through  a  one-on-one 
facilitated  development  session.  In  the  one-on-one  session  a  facilitator  (who  is  an  expert 
in  feedback  interpretation)  is  available  to  provide  helpful  planning  and  development 
support  (Bracken,  1994;  Morical,  1999;  Vinson,  1996).  The  facilitator  can  help  the 
leaders  read  and  interpret  reports,  develop  and  write  action  plans,  guide  discussion  of 
feedback  results  with  raters,  and  review  progress  (Bracken,  1994). 

London  and  Smither  (1995)  found  that  negative  and  discrepant  feedback  is 
usually  seen  as  inaccurate,  less  useful,  and  related  to  negative  reactions.  Therefore, 
developmental  feedback  program  administrators  may  need  to  tailor  follow-up  activities 
for  those  who  receive  negative  feedback.  Furthermore,  leaders  will  not  act  on  feedback 
they  do  not  understand.  A  one-on-one  facilitated  session  is  an  excellent  way  to  combat 
these  problems  and  to  ensure  leaders  gain  the  most  benefit  from  the  feedback. 

If  the  organization  does  not  have  the  resources  to  support  one-on-one  facilitated 
sessions  but  still  wants  some  personal  interaction  during  action  planning,  it  can  use 
development  workshops  (Alimo-Metcalfe,  1998;  Bracken,  1994).  The  workshops  can 
help  prepare  the  leader  to  receive  feedback,  assist  in  the  interpretation  of  reports,  and 
provide  guidance  for  action  planning.  Workshops  also  provide  personal  interaction  in  a 
group  setting,  which  can  allow  leaders  to  exchange  ideas.  Workshops  provide  the  same 
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functions  as  a  one-on-one  facilitated  development  session  except  they  do  not  provide  a 
tailored,  individual  experience. 

The  final  option  an  organization  has  for  action  planning  is  the  development 
workbook  or  self  study  guide  (Bracken,  1994;  Church,  1995).  The  workbook  can  assist 
leaders  in  analyzing  the  reports,  extracting  strengths  and  weaknesses  from  the  feedback, 
and  developing  action  plans.  These  workbooks  should  include  formats  for  conducting 
feedback  meetings,  suggestions  on  how  to  conduct  feedback  meetings,  suggested  time 
lines  for  events,  and  listings  of  internal  and  external  training  resources  to  targeted  skill 
areas  (Bracken,  1994).  While  the  workbook  is  not  as  tailored  to  the  individual  as  the 
facilitated  session  or  workshop,  it  can  be  practical  for  larger  organizations. 

Whichever  option  the  organization  chooses  to  use  to  help  leaders  formulate  their 
action  plans,  it  must  ensure  that  follow-up  occurs,  as  follow-up  is  the  key  to  skill 
development  and  change.  In  order  to  ensure  they  develop  better  skills,  leaders  should 
review  plans  and  progress  quarterly,  obtain  input  into  their  action  plans  from  co-workers 
during  feedback  meetings,  and  receive  continuous  coaching  and  feedback  (Hazucha  et  al., 
1993).  The  action  planning  process  is  key  to  goal  setting,  which  leads  to  behavior 
change. 

Developmental  Feedback  Summary 

Effective  developmental  feedback  programs  will  provide  the  tools  necessary  for 
the  leader  to  move  through  the  four  stages  of  the  feedback  process.  To  ensure  leaders  are 
given  the  tools  to  improve  their  leadership  behaviors  (i.e.,  change  behavior),  effective 
developmental  feedback  programs  provide  three  functions:  (a)  feedback  collecting,  (b) 
results  reporting,  and  (c)  action  planning.  As  stated  before,  to  ensure  the  leader  gets 
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reliable  and  accurate  information,  the  feedback  instrument  chosen  for  the  developmental 
feedback  program  must  be  reliable  and  valid  (Bracken,  1994;  Fletcher,  Baldry,  & 
Cunningham-Snell,  1998;  Morical,  1999;  Vinson,  1996)  and  should  have  been  developed 
based  on  statistical  methods  (Vinson,  1996).  The  items  in  the  instrument  should  focus  on 
behaviors  (not  traits)  that  are  important  to  the  organization  and  highlight  the 
organization’s  values  (Bracken,  1994;  London  &  Smither,  1995;  Morical,  1999).  The 
web-based  developmental  feedback  program  in  this  study  tested  a  feedback  instrument 
that  was  based  on  two  previously  used  feedback  instruments. 

Theory  Behind  and  Developmen  t  of  the  Feedback  Instrumen  ts 
The  Leadership  Practices  Inventory  (LPI)  has  been  used  extensively  to  assess 
leadership  behaviors.  Posner  and  Kouzes  (1988),  the  developers  of  the  LPI,  went  through 
the  necessary  steps  to  ensure  their  instrument  was  reliable  and  valid.  Patton’s  upward 
feedback  instrument  (UFI)  was  based  on  the  LPI  and  measures  leadership  at  a  more 
specific  commitment  level.  Patton  also  went  through  the  proper  steps  to  develop  the 
instrument  but  has  only  one  study  to  support  its  reliability  and  validity.  The  leadership 
commitments  inventory  (LCI),  introduced  in  this  study,  was  based  on  the  results  from 
Patton’s  (2002)  pilot  UFI  study.  It  was  developed  because  the  six  constructs  and  12 
commitments  from  the  UFI  needed  further  refinement,  Kouzes  and  Posner  (2002) 
released  an  updated  version  of  The  Leadership  Challenge,  and  there  was  a  shift  from  a 
purely  upward  feedback  instrument  to  a  360-degree  approach.  A  fourth  feedback 
instrument,  the  stakeholder  assessment,  was  also  introduced  in  this  study.  This 
instrument  was  designed  for  the  web-based  program,  measured  customer  satisfaction,  and 
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was  used  to  assess  criterion  validity  for  the  LCI.  The  following  is  a  review  of  the 
instruments. 

Leadership  Practices  Inventory 

Development  of  the  Leadership  Practices  Inven  tory.  To  develop  the  original 
Leadership  Practices  Inventory  (LPI),  Posner  and  Kouzes  (1988)  asked  managers  to 
describe  their  “personal  best  as  a  leader”  to  gain  a  qualitative  perspective  on  what  leaders 
do.  The  personal  best  incident  was  designed  to  highlight  a  time  when  the  leader  was 
responsible  for  an  extraordinary  organizational  accomplishment.  The  personal  best 
survey  consisted  of  37  open-ended  questions.  Some  examples  of  questions  were:  “What 
made  you  believe  you  could  accomplish  the  results  you  sought?”  and  “What  did  you 
learn  most  from  the  experience?”  (p.  484).  There  were  650  surveys  of  middle  to  senior 
managers  collected  along  with  38  in-depth  interviews.  The  interview  consisted  of  the 
same  type  of  questions  as  the  survey. 

These  surveys  and  interviews  were  content  analyzed  to  find  the  most  frequently 
mentioned  leadership  behaviors.  After  many  content  analyses,  five  leadership  constructs 
or  practices  surfaced.  The  practices  and  their  strategies  (called  commitments)  are  found 
in  Table  1.  Posner  and  Kouzes  (1988)  claim  that  over  80%  of  behavior  and  strategies 
reported  in  the  sample  of  leaders’  personal  best  experiences  were  represented  by  these 
constructs. 

To  develop  the  instrument  items,  they  administered  a  group  of  items  to  120  MBA 
students.  Subsequently,  to  perfonn  the  required  content  analysis,  they  went  through  an 
item-by-item  discussion  and  any  difficult,  ambiguous,  or  inconsistent  items  were 
eliminated,  replaced,  or  revised.  It  is  desirable  for  experts  in  the  field  to  perfonn  this 


Table  1 

Leadership  Practices  and  Commitments 
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Leadership  Practices 

Strategies  (Commitments) 

Challenge  the  Process 

1 .  Search  out  challenging  opportunities  to  change, 

grow,  innovate,  and  improve 

2.  Experiment,  take  risks,  and  learn  from  the 

accompanying  mistakes 

Inspire  a  Shared  Vision 

3.  Envision  an  uplifting  and  ennobling  future 

4.  Enlist  others  in  a  common  vision  by  appealing  to 

their  values,  interests,  hopes,  and  dreams 

Enable  Others  to  Act 

5.  Foster  collaboration  by  promoting  cooperative  goals 

and  building  trust 

6.  Strengthen  people  by  giving  power  away,  providing 

choice,  developing  competence,  assigning  critical 

tasks,  and  offering  visible  support 

Model  the  Way 

7.  Set  the  example  by  behaving  in  ways  that  are 

consistent  with  shared  values 

8.  Achieve  small  wins  that  promote  consistent 

progress  and  build  commitment 

Encourage  the  Heart 

9.  Recognize  individual  contributions  to  the  success  of 

every  project 

10.  Celebrate  team  accomplishments  regularly 

Note.  From  “The  Leadership  Challenge,”  by  Kouzes  and  Posner  (1995). 
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type  of  analysis  (Huck  &  Cormier,  1996)  and  in  this  case,  nine  professionals  in 
psychology,  organizational  behavior,  and  human  resource  management  familiar  with 
psychometric  issues,  the  instrument’s  conceptual  framework,  and  management  issues 
provided  feedback  on  the  items. 

After  revising  the  items,  Posner  and  Kouzes  (1988)  administered  the  items  to 
2,100  managers  and  subordinates  to  determine  internal  reliability  (consistency)  and 
construct  validity  by  analyzing  the  factor  structure.  The  proposed  instrument  lent  itself  to 
exploratory  factor  analysis  to  test  if  five  distinct  leadership  constructs  would  emerge. 

The  technique  identified  the  extent  to  which  each  item  was  related  to  each  factor  (or 
construct),  and  if  the  practices  were,  in  fact,  five  separate  constructs  (Dooley,  2001). 
Based  on  these  analyses,  they  rewrote  the  weak  items  and  created  the  first  version  of  the 
LPI,  which  consisted  of  30  statements  (six  for  each  practice).  There  were  two  versions, 
the  LPI-Self  and  LPI-Other.  Leaders  used  the  LPI-Self  to  perform  a  self-assessment  of 
their  own  leadership  behaviors.  Observers  (e.g.,  direct  reports)  used  the  LPI-Other  to 
assess  their  leaders’  behaviors  (Posner  &  Kouzes,  1988). 

To  incorporate  findings  from  the  vast  amount  of  data  collected  since  the  inception 
of  the  LPI,  Posner  and  Kouzes  (2000)  published  a  second  version.  In  this  version,  the 
response  format  was  changed  from  a  5-point  scale  to  a  10-point  scale  and  some  of  the 
statements  were  revised.  They  also  divided  observer  scores  into  categories  (e.g., 
manager,  direct  report,  peer  or  co-worker,  and  other).  However,  the  basic  structure  of  the 
LPI  was  not  changed  and  the  categories  allowed  leaders  to  receive  more  specific 
feedback.  Similar  to  the  first  version,  the  second  version  still  measured  the  frequency 
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with  which  a  leader  engaged  in  leadership  behaviors  consistent  with  the  five  leadership 
practices. 

Since  then,  Kouzes  and  Posner  (2002)  have  published  a  new  version  of  their 
book,  The  Leadership  Challenge.  They  posited  that  while  the  content  of  leadership  had 
not  changed,  the  context  had.  Hence,  this  version  of  the  book  maintains  the  five  practices 
but  updates  the  commitment  definitions.  They  also  reordered  the  presentation  of  the 
leadership  practices  to  show  leadership  as  “a  personal  journey  of  exploration  and  then  as 
a  rallying  of  others”  (p.  xxvi).  The  five  practices  and  updated  commitments  are  found  in 
Table  2.  There  were  some  semantic  differences  in  the  commitment  definitions  and  of 
particular  interest  were  the  commitments  defining  the  “Model  the  way”  practice.  The 
original  commitment  definitions  were  (a)  set  the  example  by  behaving  in  ways  that  are 
consistent  with  shared  values  and  (b)  achieve  small  wins  that  promote  consistent  progress 
and  build  commitment.  The  commitments  now  read  (a)  find  your  voice  by  clarifying 
your  personal  values  and  (b)  set  the  example  by  aligning  actions  with  shared  values. 

“Find  your  voice”  replaced  the  “Achieve  small  wins”  commitment  and  was  put  in  place 
to  show  that  communication  is  key  to  becoming  a  credible  leader.  Once  a  leader  finds  his 
or  her  voice,  he  or  she  must  then  set  the  example  to  earn  and  sustain  credibility.  This  was 
the  only  major  change  to  the  practices  and  commitments  in  the  new  book. 

LPI  reliability.  Internal  reliability  or  consistency  focuses  on  the  degree  to  which 
the  same  characteristics  are  being  measured  and  test-retest  reliability  focuses  on  the 
consistency  of  an  instrument  over  time  (Huck  &  Cormier,  1996).  The  LPI  has  been 
extensively  tested  for  both. 


Table  2 

Leadership  Practices  and  Commitments  (Updated) 
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Leadership  Practices 

Commitments 

Model  the  Way 

1.  Find  your  voice  by  clarifying  your  personal  values. 

2.  Set  the  example  by  aligning  actions  with  shared 

values. 

Inspire  a  Shared  Vision 

3.  Envision  the  future  by  imagining  exciting  and 

ennobling  possibilities. 

4.  Enlist  others  in  a  common  vision  by  appealing  to 

shared  aspirations. 

Challenge  the  Process 

5.  Search  for  opportunities  by  seeking  innovative  ways 

to  change,  grow,,  and  improve. 

6.  Experiment  and  take  risks  by  constantly  generating 

small  wins  and  learning  from  mistakes. 

Enable  Others  to  Act 

7.  Foster  collaboration  by  promoting  cooperative  goals 

and  building  trust. 

8.  Strengthen  others  by  sharing  power  and  discretion. 

Encourage  the  Heart 

9.  Recognize  contributions  by  showing  appreciation 

for  individual  excellence. 

10.  Celebrate  the  values  and  victories  by  creating  a 

spirit  of  community. 

Note.  From  “The  Leadership  Challenge,”  by  Kouzes  and  Posner  (2002). 
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The  first  version  of  the  LPI  was  originally  administered  to  2,876  managers  and 
executives  and  their  subordinates.  Internal  reliability  estimates  (using  Cronbach’s  alpha) 
for  each  of  the  five  practices  were  above  .70  for  LPI-Self  respondents  (iV=708)  and  above 
.79  for  LPI-Other  respondents  (7V=2, 168) — exceeding  the  cutoff  value  of  .70  (Huck  & 
Cormier,  1996).  Test-retest  reliabilities  for  each  practice  were  above  .93  (iV=57;  Posner 
&  Kouzes,  1988).  Thus,  all  the  reliabilities  for  the  LPI  met  standards  and  it  can  be 
considered  a  reliable  instrument  from  the  data  collected. 

Further  studies  by  Kouzes  and  Posner  (1995)  have  added  to  the  internal 
consistency  evidence.  Overall,  almost  44,000  respondents  have  completed  the  LPI  (Self 
and  Other  [now  called  Observer]).  Internal  reliability  estimates  (using  Cronbach’s  alpha) 
for  the  five  practices  have  been  above  .71  for  the  LPI-Self  respondents  (7V=6,65 1)  and 
above  .82  for  the  LPI-Observer  respondents  (iV=37,248).  The  second  version  of  the  LPI 
reported  reliability  results  that  were  consistent  with  the  first  version.  The  version  had  an 
overall  sample  of  17,908  respondents.  Internal  reliability  estimates  (Cronbach’s  alpha) 
were  above  .75  in  each  response  category  across  all  of  the  practices  (Posner  &  Kouzes, 
2000).  LPI  scores  have  also  remained  stable  over  time  period  comparisons  that  were 
conducted  on  participants  in  The  Leadership  Challenge  Workshop  every  two  years  since 
1987,  showing  consistency  across  the  practices. 

Other  studies  have  demonstrated  the  reliability  of  the  LPI  as  well.  Fields  and 
Hero  Id  (1997)  used  the  LPI  to  investigate  “whether  the  broader  dimensions  of 
transformational  and  transactional  leadership  can  be  inferred  from  subordinates  reports  of 
leadership  behaviors  using  instruments  not  specifically  designed  for  this  purpose.”  They 
reported  reliability  estimates  (Cronbach’s  alpha)  ranging  between  .82  and  .92  (/V=  1 ,892). 
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In  a  study  to  investigate  optimism  and  pessimism  in  business  leaders,  Wunderley,  Reddy, 
and  Dember  (1998)  administered  the  LPI  to  more  than  2,000  managers  and  their 
subordinates.  Coefficient  alphas  for  this  study  were  above  .70  for  the  LPI-Self  and  above 
.81  for  the  LPI-Observer  responses.  Collectively,  the  results  provided  by  Posner  and 
Kouzes  (1988,  1995,  2000)  and  other  studies  provided  evidence  that  the  LPI  is  a  reliable 
instrument. 

LPI  validity.  Since  the  original  version  of  the  LPI,  Kouzes  and  Posner  (1995) 
posited  the  LPI  has  been  tested  for  construct  validity  using  factor  analyses  across 
numerous  samples.  Aggregate  scores  of  all  respondents  (N=43,899)  showed  a  few  factor 
cross  loadings  but  for  the  most  part  they  obtained  five  factors,  consistent  with  the 
subscales  of  the  LPI.  Validity  assessments  for  the  second  version  of  the  LPI  indicated 
that  some  statements  loaded  on  more  than  one  factor  but  the  results  provided  support  that 
leadership  behaviors  are  measured  by  the  five  practices  (Posner  &  Kouzes,  2000). 

Other  studies  have  found  similar  results.  The  study  by  Fields  and  Herold  (1997) 
found  that  while  each  of  the  practices  could  be  considered  a  different  construct,  the 
practices  did  correlate  and  may  measure  some  of  the  same  behaviors.  Carless  (2001) 
claimed  that  the  LPI  had  weak  discriminant  validity  because  the  practices  (constructs) 
correlated  with  each  other,  indicating  that  there  might  not  be  five  separate  constructs. 
Furthermore,  the  Patton  study  (2002)  also  found  cross  loadings  and  high  correlations 
among  the  five  practices.  These  results  indicated  there  could  be  concerns  for  those 
looking  to  improve  their  leadership  skills.  The  leadership  practices  form  a  relatively 
general  model.  Therefore,  this  model  may  be  too  general  and  not  provide  the  concise 
information  a  leader  needs  to  improve  his  or  her  leadership  skills.  In  other  words,  a 
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simpler  model  with  fewer,  more  distinct  constructs  could  provide  more  concise 
information  and  give  better  feedback  for  leadership  improvement. 

Posner  and  Kouzes  (1988)  also  tested  the  criterion-related  validity  of  the  LPI, 
concurrently  and  predictively.  In  general,  criterion-related  validity  involves  correlating 
an  existing  measure  that  is  accepted  as  an  indicator  of  the  target  behavior  (criterion)  with 
the  new  measure.  Posner  and  Kouzes  (1988)  tested  this  concurrently  using  a  leadership 
effectiveness  scale,  which  was  administered  to  several  samples  that  simultaneously 
completed  the  LPI.  This  instrument  was  designed  to  detennine  how  much  the 
respondents  were  satisfied  with  the  leadership  provided  by  the  manager,  among  other 
things.  To  demonstrate  LPI  validity,  Posner  and  Kouzes  hypothesized  a  relationship 
between  a  leader’s  effectiveness  and  his  or  her  leadership  behaviors.  They  used  a 
stepwise  regression  analysis  with  leadership  effectiveness  as  the  dependent  variable  and 
the  five  leadership  practices  as  the  independent  variables.  Using  only  responses  from  the 
LPI-Other  (to  avoid  self  report  bias),  the  results  revealed  a  highly  significant  regression 
equation  (F=3 1 8.9,  p<. 0001)  and  explained  55%  (adjusted  R  =  .76)  of  the  variance 
around  subordinates’  assessments  of  their  leaders’  effectiveness.  Thus,  a  high  score  on 
the  LPI  probably  means  that  a  leader’s  subordinates  consider  him  or  her  an  effective 
leader. 

Posner  and  Kouzes  (1988)  tested  predictive  validity  by  determining  how  well  the 
LPI  scores  differentiated  between  high  and  low  perfonning  leaders.  To  do  this,  they 
separated  the  lowest  and  highest  third  of  the  managers  on  the  LPI-Other  leader 
effectiveness  scale  to  form  low  and  high  perfonner  categories.  They  conducted  a 
discriminant  analysis  using  85%  of  the  sample  of  LPI-Other  respondents  as  the 
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discriminant  function.  The  remaining  respondents  were  used  as  a  hold  out  sample.  In 
this  case,  the  discriminant  function  correctly  classified  92.62%  of  the  known  cases  and 
77.78%  of  the  hold  out  respondents.  The  LPI  was  also  able  to  correctly  classify 
respondents  significantly.  Consequently,  the  LPI  scores  could  be  used  (with  some 
accuracy)  to  differentiate  between  high  and  low  performing  leaders. 

Other  studies  (Fields  &  Herold,  1997;  Wunderley  et  ah,  1998)  have  used  the  LPI 
to  provide  evidence  of  the  presence  of  other  traits  in  leaders  apart  from  those  specific  to 
the  five  practices.  Fields  and  Herold  (1997)  investigated  whether  or  not  subordinates 
could  distinguish  between  transactional  and  transformational  leadership  behaviors  using 
the  LPI.  This  study  was  interpreted  two  different  ways  and  provided  evidence  of  both 
convergent  and  predictive  validity.  Leaders  that  were  rated  high  on  the  LPI  would  also 
be  expected  to  score  high  on  a  transformational  and  transactional  scale.  The  study 
showed  that  this  relationship  existed,  and  thus  provided  some  evidence  of  convergent 
validity.  Furthennore,  the  results  indicated  some  level  of  predictive  validity.  A  leader 
that  engages  in  the  behaviors  measured  by  the  LPI  might  be  expected  to  fulfill 
transfonnational  and  transactional  roles  more  effectively  in  the  future.  Therefore, 
transformational  and  transactional  behaviors  could  be  predicted  by  the  LPI. 

Wunderley  et  al.  (1998)  hypothesized  that  LPI-Observer  ratings  would  correlate 
positively  with  a  leader’s  level  of  optimism  and  negatively  correlate  with  a  leader’s  level 
of  pessimism.  While  the  relationships  were  generally  weak,  their  study  found  all 
correlations  for  optimism  were  positive  while  all  correlations  for  pessimism  were 
negative.  “Inspiring  a  shared  vision”  and  “Encouraging  the  heart”  correlated 
significantly  with  optimism.  Therefore,  they  concluded  from  their  study  that  leaders’ 
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optimism  might  have  some  positive  effect  on  their  subordinates’  perceptions  of  their 
leadership  behaviors,  which  provided  evidence  of  convergent  validity. 

Sources  of  potential  contamination.  Social  desirability  was  a  potential  source  of 
contamination  and  was  addressed  by  Posner  and  Kouzes  (1988).  Social  desirability 
explains  a  person’s  tendency  to  complete  items  in  a  socially  desirable  way.  People 
occasionally  complete  items  in  a  manner  that  gives  a  preferred  image  (Dooley,  2001). 
This  problem  could  surface  on  the  LPI  due  to  the  nature  of  the  measured  constructs. 
Rating  yourself  or  somebody  else  a  bad  leader  is  not  socially  desirable.  Leaders  might 
complete  their  self-assessments  in  a  manner  that  makes  them  look  good,  especially  if  they 
know  their  boss  might  see  the  results.  Direct  reports  rating  their  leaders  are  even  more 
susceptible  to  this  phenomenon  because  they  may  not  believe  their  responses  are 
anonymous  and  might  respond  positively  to  avoid  reprisal.  Therefore,  the  LPI  may  result 
in  skewed  data  due  to  positive  feedback  for  a  leader  who,  in  reality,  may  not  actually  be 
perceived  as  a  very  good  leader.  As  a  result,  the  feedback  does  not  adequately  reflect  the 
leader’s  behavior. 

Posner  and  Kouzes  (1988)  tested  the  extent  to  which  social  desirability  might 
explain  the  results  using  the  Marlowe-Crowne  Personal  Reaction  Inventory  and 
determined  that  none  of  the  correlations  were  statistically  significant  (/?>.05).  Thus,  there 
was  evidence  that  those  completing  the  LPI  did  not  influence  their  responses  in  a  socially 
desirable  manner. 

There  also  might  be  the  chance  that  LPI  scores  might  not  generalize  to  other 
groups,  samples,  or  settings  (Dooley,  2001).  Posner  and  Kouzes  (2002)  have  compared 
scores  across  individual  differences,  settings,  genders,  functional  disciplines,  ethnic 


25 


backgrounds,  and  cultures.  Overall,  they  found  few  statistical  differences  or  interaction 
effects  between  the  groups,  providing  evidence  that  LPI  results  could  be  generalized 
across  many  groups. 

Weaknesses  of  the  Leadership  Practices  Inven  tory.  Leadership  continues  to  be  a 
difficult  concept  to  define,  which  leaves  the  LPI  open  to  a  multitude  of  criticism.  For 
instance,  some  might  not  agree  that  the  LPI  measures  the  correct  leadership  behaviors. 
The  experts  who  determined  the  items  for  the  LPI  could  define  leadership  differently  than 
someone  else.  Thus,  the  items  may  not  reflect  leadership  in  its  totality,  the  practices 
might  have  some  ambiguity,  and  the  scales  may  measure  some  of  the  same  behaviors. 
Evidence  of  this  was  discussed  earlier  in  the  validity  section.  However,  the  LPI’s  items 
were  developed  through  a  systematic  content  analysis;  and  even  though  this  method  is 
highly  subjective  the  procedures  used  provide  evidence  that  the  appropriate  behaviors 
were  tapped. 

Most  of  the  reliability  and  validity  evidence  presented  in  this  paper  were  from 
studies  completed  by  Posner  and  Kouzes  (1988,  1995,  2000)  themselves.  Thus,  there 
may  be  the  chance  that  someone  might  interpret  the  studies  as  biased.  This  problem 
could  be  addressed  with  independent  studies  by  other  researchers  but  the  proprietary 
nature  of  the  LPI  may  discourage  further  studies. 

The  final  weakness  highlighted  is  the  fact  that  the  LPI  is  inflexible  and  expensive. 
Organizations  cannot  tailor  it  to  meet  their  exact  needs  and  the  cost  may  prevent  some 
companies  from  using  it.  Therefore,  this  study  also  highlights  two  instruments  (Upward 
Feedback  Instrument  and  Leadership  Commitments  Inventory)  that  were  developed  as  a 


result  of  these  factors. 
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Upward  Feedback  Instrument 

Development  of  the  upward  feedback  instrument.  To  develop  the  Upward 
Feedback  Instrument  (UFI),  Patton  (2002)  performed  a  content  analysis  of  Kouzes  and 
Posner’s  (1995)  existing  leadership  commitments.  Subsequently,  a  five-item  scale  was 
created  for  each  of  the  10  commitments.  The  sponsoring  organizations  felt  that  an  added 
practice  (i.e.,  Have  fun)  was  also  an  important  leadership  behavior.  Therefore,  the  “Have 
fun”  practice  and  five  item  scales  for  its  commitments  were  created  from  critical  incident 
responses.  This  process  was  similar  to  Posner  and  Kouzes’s  (1988)  creation  of  the  LPI 
items  using  the  personal  best  incident  responses.  The  “Have  fun”  commitments  were 
named  (a)  allow  humor  to  reduce  stress  and  boredom  and  (b)  promote  fun  activities  to 
relax  and  unwind.  After  creating  the  new  commitments,  the  research  team  performed 
more  content  analysis  for  all  12  scales  looking  for  bias  or  errors  and  revising  the  items  as 
necessary.  These  12  scales  (60  items)  were  subsequently  administered  to  the  sample. 

Upward  feedback  instrument  reliability.  At  the  present  time,  Patton  (2002)  is  the 
only  researcher  to  have  used  the  UFI.  During  this  pilot  study,  the  UFI  was  administered 
to  Air  Force  leaders  (N=  85)  and  their  subordinates  (A=64 1 )  that  worked  within  the 
sponsoring  organizations.  The  UFI  measured  the  extent  to  which  leadership  behaviors 
were  observed  using  a  7-point  Likert  scale  where  l=Not  observed  and  7=Almost  always 
observed.  Internal  reliability  (Cronbach’s  alpha)  estimates  for  the  12  scales  ranged  from 
.87  to  .91,  providing  evidence  of  reliability  (Huck  &  Connier,  1996). 

Test-retest  reliability  was  measured  using  Pearson’s  correlation  coefficients  for 
the  commitment  scales.  No  test-retest  time  period  was  given.  The  original  scale  means 
did  not  correlate  highly  with  the  retest  means  and  therefore  did  not  show  a  high  degree  of 


27 


stability  over  time.  The  low  retest  response  rate  (W=28)  may  have  caused  sampling  error, 
and  at  this  point  no  conclusions  can  be  drawn  about  the  actual  stability  of  the  UFI  over 
time. 

Upward  feedback  instrument  validity.  To  establish  the  construct  validity  of  the 
instrument,  confirmatory  factor  analyses  were  used  to  confirm  the  underlying  latent 
structure  for  the  commitment  scales.  The  research  team  was  interested  in  whether  or  not 
the  UFI  measured  Kouzes  and  Posner’s  (1995)  five  leadership  practices  and  the  sixth 
practice  “Have  fun.” 

Using  a  sample  of  391  employees  of  the  sponsoring  organizations,  nested 
confirmatory  factor  analyses  provided  evidence  that  a  six-factor  (practice)  model 
provided  the  best  explanation  of  the  underlying  latent  structure  of  the  12  commitment 
scales.  However,  similar  to  the  LPI,  a  convergent  validity  analysis  revealed  many  cross 
loadings  across  the  practices,  providing  evidence  that  the  scales  may  be  measuring 
constructs  that  overlap  considerably.  Cross  loadings  were  most  evident  with  the 
“Challenge  the  process”,  “Inspire  a  shared  vision”,  and  “Model  the  way”  commitments. 
Therefore,  the  convergent  validity  of  the  UFI  is  suspect  and  may  need  to  be  re-evaluated. 
Moreover,  there  was  no  criterion  validity  analysis  discussed  in  the  pilot  study. 

Sources  of  potential  contamination.  Because  the  UFI  is  based  on  the  LPI,  the 
same  sources  of  contamination  exist  (i.e.,  social  desirability  and  generalizability).  No 
discussion  of  how  the  researcher  compensated  for  social  desirability  was  found  in  the 
literature.  Furthermore,  the  UFI  was  not  compared  across  different  settings,  genders, 
functional  disciplines,  ethnic  backgrounds,  and  cultures,  questioning  the  extent  to  which 
the  UFI  results  can  be  generalized  across  many  groups. 
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Weaknesses  of  the  upward  feedback  instrument.  The  weaknesses  of  the  UFI  are 
similar  to  the  LPI  in  that  leadership  is  a  difficult  subject  to  define.  The  greatest  weakness 
of  the  UFI,  however,  is  the  lack  of  empirical  support  for  its  use.  It  simply  has  not  been 
field  tested  to  the  extent  the  LPI  has.  The  results  may  improve  and  the  instrument  may 
show  more  validity  as  the  sample  size  increases.  The  validity  of  an  instrument  is 
demonstrated  over  time,  and  provides  basis  for  the  current  study.  . 

Leadership  Commitments  Inventory 

The  Leadership  Commitments  Inventory  (LCI)  was  the  instrument  fielded  with 
this  study’s  web-based  developmental  feedback  program.  White  (2003)  modeled  and 
developed  the  LCI  using  the  research  conducted  by  Kouzes  and  Posner  (1995,  2002)  and 
Patton  (2002).  It  maintained  the  original  five  practices  from  Kouzes  and  Posner’s  LPI, 
and  the  sixth  practice  from  Patton’s  UFI.  Three  significant  factors  led  to  the 
development  of  the  new  instrument. 

First,  based  on  the  content  validity  and  factor  analysis  from  Patton’s  (2002)  pilot 
UFI  study,  the  six  constructs  and  12  commitments  needed  further  refinement. 

Specifically,  the  12  commitments  were  not  viewed  as  distinctive  constructs  and  the 
leadership  concepts  overlapped  based  on  the  pilot  UFI  items.  Due  to  the  high  inter¬ 
correlations  among  the  items  on  the  UFI,  each  practice  and  commitment  was  newly 
defined,  which  led  to  over  half  of  the  pilot  questions  being  modified  or  deleted  (White, 
2003). 

Secondly,  in  September  2002,  Kouzes  and  Posner  released  an  updated  version  of 
The  Leadership  Challenge.  The  most  significant  change  in  their  updated  version  was  the 
revision  of  the  “Model  the  way”  practice.  Previously,  “Model  the  way”  was  captured  by 
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the  two  commitments  set  the  example  and  achieve  small  wins.  They  revised  the  practice 
so  that  achieve  small  wins  was  incorporated  into  challenge  the  process,  and  instead  they 
replaced  it  with  find  your  voice.  In  addition  to  modifying  the  model  practice,  they  also 
re-ordered  the  practices  in  the  book,  to  better  reflect  the  leadership  development  process. 
The  new  order  of  the  practices  is:  (a)  Model  the  way,  (b)  Inspire  a  shared  vision,  (c) 
Challenge  the  process,  (d)  Enable  others  to  act,  and  (e)  Encourage  the  heart  (Kouzes  and 
Posner,  2002).  Based  on  the  research  by  Kouzes  and  Posner ,  find  your  voice  was  revised 
for  the  LCI  to  read  shares  personal  values  (White,  2003). 

The  third  factor  that  led  to  the  new  LCI  was  a  shift  from  a  purely  upward 
feedback  instrument  to  a  360-degree  approach.  While  the  UFI  focused  on  feedback  from 
subordinates  only,  the  LCI  focused  on  feedback  from  subordinates,  peers,  and  team 
members  in  an  attempt  to  replicate  a  360-degree  approach.  The  resulting  items  written  to 
measure  each  leadership  behavior,  focused  on  the  team  viewpoint  rather  than  the 
subordinate  viewpoint  (White,  2003). 

Development  of  the  leadership  commitments  inventory.  Based  on  the  content 
validity  and  factor  analysis  from  Patton’s  (2002)  pilot  UFI  study,  White  (2003) 
determined  the  six  constructs  and  12  commitments  needed  further  refinement.  As  a 
result,  a  definition  for  each  practice  and  commitment  identified  in  Patton’s  (2002)  pilot 
instrument  was  developed.  While  defining  each  practice  and  commitment,  the  goal  was 
to  capture  the  essence  of  Kouzes  and  Posner’s  (2002)  definitions  while  including 
concepts  from  other  leadership  experts.  These  definitions  established  the  groundwork 
for  the  content  analysis. 
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In  an  effort  to  develop  items  that  measured  their  intended  content  domain, 

White’s  (2003)  first  objective  was  to  develop  new  items  that  reflected  the  updated 
definitions.  To  do  this,  she  used  the  pool  of  60  items  from  Patton’s  (2002)  UFI.  Based 
on  the  formal  definitions  for  the  practices  and  commitments,  33  of  the  original  60  UFI 
items  were  revised  or  discarded.  Thirty-nine  new  items  were  written  and  those  items 
were  sorted  by  a  sample  of  military  officers  (N=  17).  The  sorting  exercise  produced  a 
pool  of  48  items. 

White  (2003)  conducted  a  second  content  analysis  to  analyze  five  new  items  that 
were  developed  as  a  result  of  the  change  to  the  “Model  the  way”  commitment,  shares 
personal  values.  This  analysis  also  served  to  validate  one  other  rewritten  item  from  the 
first  analysis.  A  similar  sample  of  military  officers  (A=19)  attempted  to  match  these 
items  to  their  respective  commitments.  The  five  new  items  representing  shares  personal 
values  were  all  properly  categorized.  Two  previously  validated  items  were  not  properly 
categorized  and  were  subsequently  rewritten. 

Following  the  two  content  analyses,  a  four-item  scale  was  created  for  each  of  the 
12  commitments.  These  12  scales  (48  items)  were  administered  to  a  sample  using  the 
web-based  developmental  feedback  program  that  was  developed  in  this  study. 

Stakeholder  Assessment 

The  stakeholder  assessment  was  designed  to  measure  customer  satisfaction  with  a 
leader’s  organization  and  was  to  be  administered  with  the  web-based  developmental 
feedback  program,  alongside  the  LCI.  The  results  were  to  be  used  to  provide  criterion 
validity  evidence  for  the  LCI. 
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The  stakeholder  assessment  was  designed  to  assess  customer  satisfaction  of  a 
leader’s  staff  or  work  group  and  provide  criterion  validity  evidence  for  the  LCI  under  the 
following  assumption.  An  effective  supervisor  influences  the  products  and  services  that  a 
customer  receives  from  his  or  her  staff  or  work  group.  Therefore,  under  this  assumption, 
a  leader  who  receives  high  scores  on  the  LCI  should  also  receive  high  scores  on  the 
stakeholder  assessment. 

To  measure  a  customer’s  satisfaction  with  the  organization,  the  stakeholder 
assessment  needed  to  assess  a  set  of  dimensions  that  could  be  generalized  across  a  variety 
of  work  groups  that  provide  an  array  of  products,  services,  and  information.  It  needed  to 
assess  the  performance  of  the  staff  or  work  group  rather  than  the  supervisor  because  some 
customers  may  interact  with  the  supervisor  while  others  interact  with  specific  points  of 
contact  (i.e.,  subordinates).  So  the  assessment  needed  to  evaluate  the  quality  of  both  the 
staff  or  work  group  and  the  services  and  information  the  work  group  provides  customers. 

The  dimensions  to  measure  the  performance  of  the  staff  or  work  group  and  the 
quality  of  the  information  and  services  provided  were  extracted  from  an  analysis  of 
customer  satisfaction  literature  (Davis,  1999;  Hayes,  1992;  Levitt,  1972;  Naumann  & 
Jackson,  1999;  Schneider  &  Bowen,  1999).  The  service  dimensions  were  quality  of 
information,  products,  and  services;  responsiveness;  professionalism;  and  overall 
satisfaction. 

The  quality  of  information,  products,  and  services  dimension  was  defined  as  the 
extent  to  which  the  information  received  was  clear,  free  from  error,  up-to-date,  and 
complete.  It  further  measured  the  quality  of  the  information,  products,  and  services 
received  by  the  customer.  Responsiveness  was  the  extent  to  which  the  products  and 
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services  were  delivered  when  needed  and  free  from  error.  It  further  measured  how  well 
the  work  group  adapted  to  the  customer  requirements.  Professionalism  measures  the 
extent  to  which  the  staff  members  were  knowledgeable,  courteous,  and  caring  when 
providing  needed  products,  services,  and  information.  Finally,  overall  satisfaction  was 
defined  as  the  extent  to  which  the  customer  was  pleased  with  the  needed  products, 
services,  and  information  and  would  freely  choose  this  work  group’s  products  if  given  a 
choice  of  providers. 

The  stakeholder  assessment  consists  of  24  items.  Six  items  measure  quality  of 
information,  products,  and  services;  seven  items  measure  responsiveness;  four  items 
measure  professionalism;  and  four  items  measure  overall  satisfaction.  There  are  also  two 
open-ended  items  that  allow  the  customer  to  identify  things  the  work  group  does  well  and 
needs  to  improve.  The  last  item  of  the  assessment  is  an  open-ended  item  that  allows  the 
customer  to  provide  feedback  for  the  leader.  Reliability  or  validity  evidence  for  the 
assessment  and  its  scales  has  not  been  compiled  and  reported. 

From  Paper-and-Pencil  to  Computer-and-Keyboard 

Web-based  surveys  are  an  extremely  promising  method  for  conducting  research 
(Mertler,  2002).  First  and  foremost,  they  are  an  excellent  resource  for  obtaining  and 
processing  large  amounts  of  data  and  they  can  be  distributed  over  the  Internet  via  a  web 
site  or  e-mail  (Mertler,  2002;  Sheehan  &  Hoy,  1999).  Web-based  surveys  and  the 
methods  used  to  deploy  them  have  advantages,  limitations,  and  issues.  Even  so,  they  can 
be  a  viable  method  for  developmental  feedback  programs  and  have  been  used  in  this 
capacity.  This  section  highlights  the  advantages  and  limitations  of  a  web-based  survey 
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and  their  deployment  methods.  It  also  highlights  the  fact  that  organizations  can  use  the 
web  as  a  feasible  platform  for  their  developmental  feedback  programs. 

Advantages  of  Web-based  Surveys 

Web-based  surveys  have  many  advantages  over  traditional  surveys  (i.e.,  postal  or 
telephone).  Advantages  include  cost  and  time  savings,  flexibility,  geographical  reach, 
ease  of  use,  and  assurance  of  anonymity  and  confidentiality  (Clarkson,  1999;  Coomber, 
1997;  Kimball,  1998;  Mertler,  2002;  Sheehan  &  Hoy,  1999;  Sills  &  Song,  2002;  Slevin, 
1997;  Tierney,  2000;  Virtual  Surveys  Limited,  1996;  Watt,  1997).  Of  the  advantages, 
cost  and  time  savings  are  the  most  prevalent. 

Given  the  nature  of  web-based  surveys,  cost  savings  over  traditional  survey 
methods  are  inherent  (Clarkson,  1999;  Kimball,  1998;  Mertler,  2002;  Sills  &  Song,  2002; 
Sheehan  &  Hoy,  1999;  Slevin,  1997;  Tierney,  2000;  Virtual  Surveys  Limited,  1996; 

Watt,  1997).  Researchers  do  not  have  to  purchase  surveys,  copies,  postage,  nor 
envelopes.  Data  entry,  analysis,  and  reporting  are  all  automatic,  negating  the  need  for 
personnel  and  time  to  perform  these  functions.  There  are  also  no  requirements  for 
interviewers.  Watt  (1997)  posits  that  the  cost  advantage  over  traditional  surveys  is 
realized  for  surveys  with  more  than  500  respondents.  This  may  be  due  to  the  initial  start 
up  costs  for  web  design  and  hosting  (Sheehan  &  Hoy,  1999). 

Time  savings  are  also  a  key  advantage  to  the  web-based  survey  methodology. 

Time  savings  are  realized  through  quick  survey  response  collection  (Clarkson,  1999; 
Kimball,  1998;  Mertler,  2002;  Sheehan  &  Hoy,  1999;  Sills  &  Song,  2002;  Tierney,  2000; 
Watt,  1997).  The  total  time  of  survey  delivery  and  response  can  be  reduced  by  weeks 
over  the  postal  method,  allowing  researchers  to  make  quicker  decisions  (Sheehan  &  Hoy, 
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1999).  Further  time  savings  are  realized  through  automatic  data  entry,  analysis,  and 
reporting.  There  are  no  requirements  to  manually  perform  these  functions.  Automatic 
data  entry  also  reduces  the  human  error  factor  of  manual  data  entry,  saving  the  time 
required  to  re-enter  the  data  (Clarkson,  1999;  Mertler,  2002;  Sills  &  Song,  2002;  Tierney, 
2000;  Watt,  1997). 

Web-based  surveys  are  flexible  to  modify  (Clarkson,  1999;  Kimball,  1998; 
Sheehan  &  Hoy,  1999;  Sills  &  Song,  2002;  Watt,  1997).  Surveys  can  be  modified  after 
they  have  been  fielded,  which  can  prove  to  be  a  huge  advantage  over  paper  surveys.  If  a 
researcher  needs  to  change  a  question  or  item  it  can  be  done  quickly  and  without  the 
respondents’  knowledge.  For  paper  surveys,  this  cannot  be  accomplished  without 
sending  out  a  revised  set  of  surveys.  Web-based  surveys  also  have  a  lot  of  flexibility  in 
reporting  results.  Reports  can  be  tailored  to  individuals  or  groups  (Watt,  1997). 
Participants  are  able  to  choose  the  most  convenient  response  fonnat.  They  can  choose 
the  survey  over  the  web  or  print  the  survey  out  and  mail  it  in  (Coomber,  1997). 

Web-based  surveys  can  reach  a  large,  more  diverse  cross  section  of  participants 
than  traditional  surveys  because  of  the  geographical  reach  of  the  Internet  (Coomber, 

1997;  Mertler,  2002;  Sheehan  &  Hoy,  1999;  Sills  &  Song,  2002).  This  results  in  a 
greater  percentage  of  responses  from  a  diverse  group  of  respondents.  The  reach  of  web- 
based  surveys  can  prove  to  be  an  advantage  for  researchers  who  are  looking  for 
respondents  across  geographical  borders  and  for  large  organizations  with  dispersed 
personnel  (Sheehan  &  Hoy,  1999). 

Another  advantage  of  web-based  surveys  is  their  ease  of  use  (Clarkson,  1999; 
Kimball,  1998;  Mertler,  2002;  Sills  &  Song,  2002;  Slevin,  1997;  Virtual  Surveys 
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Limited,  1996).  Respondents  are  able  to  complete  and  submit  the  survey  at  the  same 
time  (Clarkson,  1999;  Slevin,  1997).  Graphics  and  multimedia  features  (e.g.,  option 
buttons,  tutorials)  also  enhance  web-based  surveys.  Explanations  and  instructions  can  be 
integrated  into  the  survey  and  accessed  by  the  respondent  when  needed  (Clarkson,  1999). 
These  features  speed  up  completion  time  and  make  the  survey  more  interesting  for  the 
respondent.  Virtual  Surveys  Limited  (1996)  found  that  these  features  could  result  in 
higher  quality  responses,  reducing  item  omission  and  response  error  and  increasing 
completeness  of  answers  in  open-ended  questions.  For  instance,  in  their  study,  a  larger 
number  of  comments  for  the  open-ended  questions  were  coded  from  surveys  completed 
on-line  than  those  completed  on  paper. 

For  some  surveys  it  is  critical  that  the  participants  and  their  responses  are 
anonymous  and  confidential  (Clarkson,  1999;  Coomber,  1997;  Kimball,  1998;  Mertler, 
2002;  Sheehan  &  Hoy,  1999;  Sills  &  Song,  2002;  Slevin,  1997).  Anonymity  and 
confidentiality  can  be  difficult  to  guarantee  and  some  respondents  may  be  wary  that  their 
answers  are  not  anonymous  (Clarkson,  1999).  Therefore,  researchers  must  demonstrate 
anonymity  to  the  respondent  (Coomber,  1997).  Researchers  can  implement  and  include  a 
security  system  to  protect  the  data  and  ease  the  mind  of  the  respondent  (Mertler,  2002). 
Most  organizations  provide  anonymity  and  confidentiality  by  using  an  outside  source  to 
process  and  analyze  the  data,  and  report  the  results  (Clarkson,  1999;  Sills  &  Song,  2002; 
Slevin,  1997).  Others  propose  using  passwords  to  control  who  responds  to  the  survey 
and  who  has  access  to  the  results  (Kimball,  1998).  When  using  e-mail  surveys, 
encryption  technology  and  assurances  of  confidentiality  may  be  required  (Sheehan  & 
Hoy,  1999). 
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Web-based  surveys  can  be  used  to  collect  data  with  greater  efficiency  and 
convenience  while  saving  time  and  money.  These  features  alone  make  the  methodology 
appealing  (Mertler,  2002).  However,  before  adopting  this  methodology  one  should  be 
aware  of  its  many  limitations. 

Limitations  of  Web-based  Surveys 

Web-based  surveys  have  three  main  limitations  or  disadvantages.  The  most 
commonly  discussed  limitation  is  sample  bias.  Technical  concerns  can  also  be  a 
limitation.  Finally,  security  concerns  with  the  Internet  leave  respondents  wary  that  their 
responses  may  not  be  anonymous. 

Studies  posit  that  sample  bias  is  caused  by  the  non-random  nature  and  self¬ 
selectivity  of  Internet  samples  (Clarkson,  1999;  Mertler,  2002;  Sills  &  Song,  2002; 
Sheehan  &  Hoy,  1999;  Tierney,  2000;  Virtual  Surveys  Limited,  1996).  Not  all  people  in 
a  proposed  population  have  access  to  the  Internet.  Furthermore,  there  is  no  worldwide 
population  list  of  Internet  users  and  not  all  people  in  the  population  are  given  the  chance 
to  complete  the  survey.  Therefore  the  general  population  is  not  a  good  population  for  a 
web-based  survey  (Watt,  1997).  Further  exacerbating  the  bias  is  self-selectivity  of 
Internet  users  to  complete  the  survey.  Combining  these  factors  indicate  that  web-based 
surveys  may  not  be  generalizable  (Clarkson,  1999;  Sheehan  &  Hoy,  1999;  Virtual 
Surveys  Limited,  1996). 

Further  threats  causing  bias  are  non-response,  multiple  responses,  and 
inappropriate  responses.  Non-response  error  can  be  caused  by  technical  problems,  timing 
of  follow-up,  confidentiality  and  anonymity  concerns,  and  misidentification  of  the  survey 
as  junk  e-mail  (Sills  &  Song,  2002;  Sheehan  &  Hoy,  1999;  Tierney,  2000;  Virtual 


37 


Surveys  Limited,  1996).  More  bias  is  introduced  when  a  respondent  completes  more 
than  one  survey  or  completes  a  survey  inappropriately  (Sheehan  &  Hoy,  1999).  Web- 
based  surveys  are  especially  susceptible  to  this  behavior  because  many  times  the 
researcher  has  no  control  over  who  visits  the  web  site.  Incentives  also  cause  multiple 
responses  and  unwanted  entries  (Tierney,  2000).  One  way  to  combat  non-response  and 
multiple  responses  is  by  using  an  e-mail  survey.  An  e-mail  survey  can  identify  non¬ 
respondents  or  multiple  responses  by  the  same  person.  However,  this  is  accomplished  at 
the  expense  of  anonymity  and  confidentiality  (Sheehan  &  Hoy,  1999). 

While  reducing  bias  is  always  a  major  concern  in  survey  research,  response  rates 
are  key  to  the  validity  of  the  survey  results  (Yun  &  Trumbo,  2000).  However,  there  is 
little  evidence  that  web-based  surveys  result  in  higher  response  rates  than  postal  surveys. 
Sheehan  and  Hoy  (1999)  reviewed  nine  studies  that  used  postal  and  e-mail  surveys.  Four 
studies  showed  that  postal  survey  achieved  higher  response  rates  than  e-mail,  three 
studies  showed  that  e-mail  achieved  higher  response  rates  than  postal,  and  two  studies 
showed  no  significant  difference  between  the  two.  Other  studies  have  found  similar 
results  (e.g.,  Yun  &  Trumbo,  2000).  Nonetheless,  there  are  certain  factors,  inherent  to 
web  surveys,  that  affect  response  rate  (e.g.,  technical  problems  and  security  concerns). 

Technical  problems  create  another  limitation  for  web-based  surveys  (Clarkson, 
1999;  Mertler,  2002;  Sills  &  Song,  2002).  Delivery  of  the  survey  instrument  can  be 
hindered  if  respondents’  Internet  browsers  are  not  compatible  with  the  survey  technology 
or  there  is  a  problem  with  the  Internet  link  to  the  survey.  Researchers  must  ensure  the 
survey  is  compatible  with  a  wide  variety  of  browsers  and  software  (Clarkson,  1999). 
Furthermore,  potential  respondents  may  not  have  Internet  or  e-mail  access  or  may  not  be 
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familiar  with  computer  use  (Clarkson,  1999;  Mertler,  2002;  Slevin,  1997).  Additionally, 
if  using  an  e-mail  survey,  the  e-mail  may  fail  to  reach  intended  recipients  (i.e.,  the  wrong 
name  was  typed  in  or  the  respondent’s  e-mail  address  was  changed). 

There  also  may  be  security  concerns  associated  with  web-based  surveys. 
Respondents  may  not  believe  their  anonymity  and  confidentiality  is  assured,  and  may 
worry  about  potential  identification  (Mertler,  2002;  Sills  &  Song,  2002).  Consequently, 
researchers  should  hire  an  outside  organization  to  collect  and  process  the  responses  and 
results,  which  can  cause  an  increase  in  cost  and  complexity  (Sills  &  Song,  2002). 

These  limitations  can  be  a  cause  for  concern  when  deciding  whether  or  not  to 
implement  a  web-based  survey.  Researchers  who  do  decide  to  use  the  web-based 
methodology  must  choose  which  deployment  method  to  use,  web  site  or  e-mail. 

Survey  Distribution:  Web  Site  or  E-mail 

There  are  two  ways  to  distribute  surveys  via  the  Internet.  Surveys  can  be  sent 
directly  to  potential  respondents  via  e-mail  or  posted  on  a  web  site  (Clarkson,  1999; 
Kimball,  1998;  Mertler,  2002;  Sheehan  &  Hoy,  1999;  Slevin,  1997;  Tierney,  2000; 
Virtual  Surveys  Limited,  1996;  Watt,  1997).  Normally,  when  using  a  web  site,  an  e-mail 
that  contains  a  link  to  the  survey  or  web  site  is  sent  to  potential  respondents  (Mertler, 
2002;  Tierney,  2000;  Watt,  1997).  Respondents  visit  the  web  site  and  choose  to  complete 
the  survey.  Given  these  choices,  organizations  can  choose  which  medium  to  use  given 
the  advantages  and  limitations  of  each. 

E-mail.  E-mail  surveys  can  be  beneficial  because  respondents  do  not  need  access 
to  the  Internet  or  a  have  a  compatible  browser;  they  just  need  an  e-mail  account  (Slevin, 
1997;  Virtual  Surveys  Limited,  1996).  For  reasons  such  as  this,  e-mail  surveys  have  a 
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tendency  to  increase  response  rates  over  web  site  surveys.  E-mail  allows  for  ease  and 
flexibility  of  responding  because  the  e-mail  survey  requires  few  steps  to  complete 
(Slevin,  1997).  Both  accessibility  and  simplicity  give  e-mail  surveys  better  response 
rates  than  web  site  surveys  (Slevin,  1997).  Furthermore,  e-mail  allows  the  researcher  to 
identify  duplicate  responses  and  non-responses.  Researchers  can  therefore  profile  non¬ 
respondents.  E-mail  also  has  financial  benefits  over  a  web  site  because  there  are  no 
design  or  web  hosting  costs  (Sheehan  &  Hoy,  1999) 

While  e-mail  surveys  have  some  advantages  over  a  web  site,  they  also  have  their 
limitations.  E-mail  questionnaires  are  usually  fonnatted  with  grids  of  questions  and  scale 
responses,  which  are  not  visually  attractive  (Watt,  1997).  This  format  can  also  make 
them  difficult  to  complete  (Virtual  Surveys  Limited,  1996).  E-mail  surveys  do  not  allow 
incremental  checks  for  validity  of  the  inputs  until  the  survey  has  been  completed  and 
submitted.  For  instance,  if  the  researcher  finds  missing  responses,  he  or  she  must  return 
the  survey  to  the  respondent  for  the  required  infonnation  (Slevin,  1997;  Watt,  1997).  E- 
mail  addresses  frequently  change  and  respondents  may  not  be  reached  (Virtual  Surveys 
Limited,  1996).  As  stated  before,  e-mail  allows  the  researcher  to  identify  duplicate 
responses  and  non-responses,  eliminating  respondent  anonymity.  Finally,  unsolicited 
surveys  can  be  seen  as  aggressive  and  violating  netiquette  (Coomber,  1997;  Sheehan  & 
Hoy,  1999). 

Web  site.  As  stated  above,  e-mail  is  not  the  only  web-based  choice  to  distribute 
surveys.  Surveys  posted  on  web  sites  also  have  their  advantages  and  limitations.  Given 
that  potential  respondents  have  access  to  the  Internet,  surveys  posted  on  web  sites  are  a 
more  lucrative  medium  than  e-mail  surveys  (Clarkson,  1999).  The  most  optimum 
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method  is  to  contact  the  respondent  by  e-mail  and  ask  him  or  her  to  visit  a  web  site  to 
complete  the  survey.  This  speeds  response  time  and  provides  better  questionnaire 
interface  than  e-mail  (Virtual  Surveys  Limited,  1996).  Web  site  surveys  are  easier  to 
complete  because  they  have  option  buttons.  Graphics,  images,  animation,  and  links  can 
be  used  to  make  the  survey  faster  and  more  interesting,  resulting  in  higher  quality  data. 
These  surveys  also  have  the  ability  to  check  for  valid  responses  as  the  respondent 
completes  the  survey.  If  bad  data  are  entered,  the  respondent  receives  an  error  message 
that  instructs  them  to  re-enter  the  data  (Slevin,  1997).  Database  operations  and  queries 
can  be  programmed  to  adapt  the  survey  results  to  virtually  any  reporting  need  the 
researcher  may  have  (Watt,  1997). 

Surveys  posted  on  a  web  site  are  not  without  limitations.  First  of  all,  respondents 
must  have  access  to  the  Internet  to  complete  the  survey  (Mertler,  2002;  Slevin,  1997; 
Virtual  Surveys  Limited,  1996).  Furthennore,  the  respondent  is  required  to  move 
through  many  steps  before  they  can  complete  the  survey  (Slevin,  1997).  Technical 
problems  with  incompatible  Internet  browsers  and  user  inexperience  can  also  eliminate 
some  respondents  (Clarkson,  1999;  Virtual  Survey  Limited,  1996).  If  the  web  site  and 
survey  contain  too  many  graphics  or  features,  the  load  time  of  the  site  is  increased  which 
has  been  know  to  produce  lower  response  rates  than  e-mail  surveys  (Clarkson,  1999; 
Slevin,  1997). 

Regardless  of  whether  an  organization  chooses  a  web  site  or  e-mail  to  deploy  a 
survey,  there  will  be  advantages  and  limitations.  Research  suggests  that  web  sites  are 
best  when  studying  large  groups  of  on-line  users  and  e-mail  surveys  are  better  for 
studying  smaller,  homogenous  on-line  user  groups  (Sheehan  &  Hoy,  1999).  Based  on  its 
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advantages,  a  web-based  (i.e.,  web  site  or  e-mail)  survey  is  a  sound  choice  when 
respondents  have  access  to  the  Internet  (Clarkson,  1999).  Therefore,  Internet-based 
surveys  should  be  used  to  target  groups  for  academic,  business,  and  employee  research. 
They  are  also  feasible  for  organizations  that  adopt  new  technology  early  (Virtual  Surveys 
Limited,  1996).  Some  organizations  have  chosen  a  technology-based  methodology  to 
implement  their  developmental  feedback  programs. 

Web-based  Developmental  Feedback  Programs 

The  Internet  is  a  viable  platfonn  for  developmental  feedback  programs.  Bracken 
and  Summers  (1998)  summarized  a  methodology  for  implementing  a  web-based 
developmental  feedback  program,  the  advantages  of  web-based  developmental  feedback, 
and  factors  to  consider  before  deploying  a  web-based  system. 

The  methodology  discussed  by  Bracken  and  Summers  (1998)  was  relatively 
simple.  The  feedback  survey  was  loaded  on  a  web  site.  The  service  provider  sent  e- 
mails  to  participants  (i.e.,  leaders)  with  instructions  and  timeframes  for  the  assessment. 
The  participants  accessed  the  web  site  and  keyed  in  their  personal  identification  and 
password.  The  leaders  then  created  a  list  of  raters  (i.e.,  peers,  subordinates,  etc.)  and 
selected  the  names  of  the  raters  from  a  drop  down  list  or  simply  entered  the  names  and 
valid  e-mail  addresses.  Raters  were  sent  an  e-mail  message  requesting  them  to  go  to  the 
web  site  to  complete  an  assessment.  At  the  end  of  the  allotted  time  window,  the  feedback 
was  collected,  collated,  and  assembled  into  reports.  The  feedback  reports  were  read  from 
the  web  site.  The  web  site  also  contained  a  link  to  an  interactive  development  planning 
system.  This  system  guided  the  participant  through  the  steps  to  identify  key  development 
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areas.  It  also  helped  the  participant  design  and  develop  an  action  plan  so  they  could  turn 
the  feedback  into  behavior  change. 

Bracken  and  Summers  (1998)  discussed  four  main  advantages  to  web-based 
developmental  feedback.  For  instance,  web-based  systems  ensured  rater  reliability  and 
confidentiality.  They  reduced  the  resources  required  to  administer  the  program  and 
allowed  people  to  redirect  their  efforts  to  value-added  activities  such  as  action  planning. 
Systems  were  capable  of  covering  a  large  geographical  area,  making  them  good  for  large, 
dispersed  organizations.  Finally,  the  cost  per  participant  was  considerably  less  than 
traditional  survey  methods  (i.e.,  paper). 

Before  deploying  a  web-based  developmental  feedback  program,  organizations 
should  review  some  important  factors.  First  of  all,  the  organization  should  have  some 
prior  developmental  feedback  experience.  This  can  reduce  administrative  burdens  and 
make  it  easier  for  participants  and  raters  to  adapt  to  the  new  technology.  Second, 
participants  and  raters  must  have  access  to  the  Internet.  The  organization  should  provide 
a  special  computer  station  for  those  who  do  not  have  access.  Third,  users  must  be  able  to 
move  through  the  process  easily.  They  should  try  to  be  patient  if  technical  problems  or 
setbacks  occur.  If  the  senior  management  of  the  organization  has  nurtured  a  technology 
supportive  culture,  the  implementation  of  the  program  will  be  easier.  Additionally,  the 
organization  should  have  some  technical  sophistication  and  adequate  infonnation 
technology  resources.  System  administrators  must  implement  features  that  ensure  full 
security  of  the  system  and  confidentiality  for  users  and  these  features  must  be 
communicated  to  the  user.  Furthermore,  there  should  be  adequate  support  during  the 
early  phases  of  the  program  (Bracken  and  Summers,  1998). 
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Summary  of  Web-based  Surveys 

Web-based  surveys  are  an  extremely  promising  method  for  conducting  research 
(Mertler,  2002).  They  have  plenty  of  advantages  for  organizations  that  are  “connected.” 
These  surveys  also  have  limitations  that  organizations  should  consider.  Despite 
limitations,  web-based  surveys  have  been  used  successfully  in  developmental  feedback 
programs  and  are  practical  for  organizations  that  have  the  proper  resources.  Given  the 
characteristics  (i.e.,  advantages  and  limitations)  of  web-based  surveys,  the  most  practical 
format  is  to  deploy  the  survey  on  a  web  site  and  advertise  to  participants  via  e-mail. 

Information  Systems  Development 

Information  systems  development  success  or  failure  can  depend  on  numerous 
factors  such  as  project  risk,  scope,  management,  design,  and  adequacy  of  resources 
(Gordon  &  Gordon,  1999).  There  were  numerous  systems  development  failures  in  the 
mid-sixties  due  to  poor  or  nonexistent  development  techniques.  In  an  attempt  to  reduce 
the  number  of  failures,  information  systems  developers  created  development 
methodologies  similar  to  engineering  systems  development  processes  (Aktas,  1987).  As 
a  result,  numerous  approaches  to  systems  development  now  exist.  The  information 
systems  development  life  cycle  (ISDLC),  which  is  prevalent  throughout  information 
systems  literature,  serves  as  a  good  guide  to  a  development  project.  The  traditional 
approach  to  the  ISDLC  however  does  not  offer  the  tools  necessary  to  successfully 
develop  an  information  system  (Aktas,  1987).  The  structured  approach  to  the  ISDLC 
provides  the  tools  and  methodology  necessary  to  guide  the  system  developer  through  the 
ISDLC  en  route  to  successful  systems  development  (Aktas,  1987).  This  section  provides 
various  examples  of  the  ISDLC,  compares  the  traditional  approach  to  systems 
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development  with  the  structured  approach,  presents  a  model  of  the  major  phases  of  the 
development  process,  and  highlights  tools  used  in  the  structured  approach  to  infonnation 
system  analysis  and  design  (i.e.,  hierarchy  charts  and  data  flow  diagrams). 

Information  Systems  Development  Life  Cycle 

A  review  of  the  literature  will  unfold  many  different  versions  of  the  information 
systems  development  life  cycle  (ISDLC).  Each  version  contains  numerous  phases  with 
different  nomenclature  but  they  all  contain  similar  components.  Peters  (1987)  presented 
many  definitions  of  the  ISDLC.  He  combined  the  commonalities  within  those  definitions 
and  described  the  ISDLC  as  a  model,  plan,  or  guide  that  is  activity  and  process  oriented 
and  is  of  use  to  both  those  managing  the  work  and  those  doing  it.  Three  life  cycle  models 
posited  during  different  time  periods  are  described  below. 

Brookes,  Grouse,  Jeffery,  and  Lawrence  (1982)  presented  an  example  of  the 
ISDLC.  Their  model  had  various  phases  with  associated  tasks.  Tasks  for  each  phase 
must  be  completed  before  moving  onto  the  next  phase.  Their  model  represented  the  life 
cycle  with  the  following  phases:  statement  of  terms  of  reference  and  specification  of 
requirements,  the  feasibility  study,  systems  analysis,  the  logical  design  of  the  new 
system,  the  physical  design  of  the  new  system,  programming,  implementation,  and  post¬ 
implementation. 

In  the  first  stage,  statement  of  terms  of  reference  and  specification  of 
requirements,  the  developer  detennines  the  user’s  requirements  and  expectations.  The 
feasibility  study  detennines  the  overall  architecture  of  the  new  system.  Systems  analysis 
provides  full  detail  of  the  existing  system  to  include  physical  flow  of  data  and 
information  through  the  system.  This  phase  also  highlights  constraints  on  improvement 
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of  the  existing  system.  The  logical  design  phase  defines  user  specification  for  the  flow  of 
data  and  information  within  the  system.  It  also  puts  focus  on  user  interface.  In  other 
words,  this  phase  defines  what  the  system  will  be  required  to  do  and  how  the  system  will 
be  implemented.  The  physical  design  phase  follows  and  here  the  developer  designs  the 
files  and  details  of  the  computer  program  modules.  The  programming  phase  consists  of 
the  actual  design  and  coding  of  the  programs.  Initial  tests  on  system  performance  are 
performed  here  as  well.  The  implementation  consists  of  the  cutover  period  where  the 
complete  system  is  implemented  and  tested  in  the  user’s  environment.  Finally,  the  post¬ 
implementation  phase  detennines  if  the  system  is  successfully  meeting  requirements.  It 
is  also  detennines  whether  or  not  the  anticipated  benefits  are  being  obtained  (Brookes  et 
al„  1982). 

Harpool,  Culp,  and  Galehorse  (1987)  proposed  another  example  of  the  ISDLC. 
They  posited  five  phases:  project  planning  and  preliminary  structured  analysis;  detailed 
structured  analysis;  system  design;  system  development  and  implementation;  and  system 
installation,  evaluation,  and  support.  Each  phase  has  required  activities  and  completion 
of  one  phase  leads  to  the  initiation  of  the  next. 

During  the  project-planning  phase  a  problem  or  need  is  identified.  Here  the 
current  system  is  evaluated  and  a  problem  definition  is  developed  before  moving  on  to 
the  next  phase.  In  the  detailed  structured  analysis  phase,  the  developer  refines  the 
problem  definition,  develops  alternative  solutions,  prepares  cost  benefit  and  risk  analysis, 
and  recommends  the  best  alternative.  Once  a  system  is  recommended  the  developer 
designs  a  new  system  to  meet  the  user’s  needs  and  objectives  during  the  system  design 
phase.  The  deliverable  products  from  this  stage  are  the  system  specifications.  Once  the 
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specifications  are  determined  a  new  system  is  developed  that  meets  design  specifications. 
During  this  system  development  and  implementation  phase  the  new  system  is  coded, 
tested,  and  preliminary  plans  for  installation  are  developed.  In  the  final  stage,  system 
installation,  evaluation,  and  support,  the  system  is  turned  over  to  the  user.  The  new 
system  is  formally  installed  and  evaluated  in  light  of  the  original  user  needs  and 
objectives  (Harpool  et  ah,  1987). 

Gordon  and  Gordon  (1999)  posited  seven  phases  of  the  ISDLC  and  briefly 
described  three  development  pathways.  They  described  the  following  phases:  needs 
assessment,  alternative  analysis,  design,  development,  implementation,  and  maintenance. 
To  move  through  the  life  cycle  stages,  system  developers  can  use  three  development 
pathways.  These  are  described  as  the  waterfall  approach,  the  spiral  approach,  and 
prototyping. 

The  needs  assessment  phase  focuses  on  gathering  data  about  user  needs  and 
ranking  needs  by  importance.  Here  it  is  determined  whether  or  not  improving  the 
existing  information  system  needs  can  satisfy  user  needs.  Another  facet  of  this  phase  is 
output,  input,  and  procedure  analysis.  This  analysis  develops  initial  plans  for  what  the 
system  can  accomplish  in  tenns  of  output  and  input,  and  what  procedures  are  required  to 
use  the  system.  Once  a  rough  plan  is  determined,  a  structured  analysis  will  provide 
diagrams  of  the  existing  and  proposed  systems  and  their  capabilities  (Gordon  &  Gordon, 
1999). 

The  alternative  analysis  phase  allows  the  system  developer  to  consider  alternate 
designs  and  the  advantages  and  disadvantages  of  each.  The  main  focus  here  is 
determining  the  trade-offs  between  designs.  The  most  prevalent  trade-off  is  user  needs 
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versus  cost.  Cost  increases  as  more  user  needs  are  satisfied.  The  phase  ends  with  a 
preliminary  design  of  the  system  (Gordon  &  Gordon,  1999). 

The  design  phase  provides  the  detailed  specifications  that  communicate  the  design 
to  programmers.  There  are  many  facets  to  design.  Interface  design  highlights  the  content 
and  form  of  the  input  and  output.  The  data  design  creates  the  model  of  data  supporting 
the  system  (i.e.,  database).  The  process  design  focuses  on  the  computational  and  logical 
processes  underlying  the  system.  The  physical  design  is  the  hardware  used  to  deliver  the 
system.  And  finally,  the  test  design  tests  the  system  to  ensure  proper  function  (Gordon  & 
Gordon,  1999). 

The  development  phase  answers  the  question  as  to  whether  or  not  the  developers 
should  purchase  a  system  or  develop  their  own.  Once  this  decision  is  made  it  is 
necessary  to  create  or  purchase  the  hardware  and  software.  Testing  also  takes  place 
during  this  phase  to  ensure  the  system  works  as  designed.  There  are  different  types  of 
tests  to  perform.  System  testing  tests  the  entire  system  under  realistic  conditions. 
Performance  testing  tests  the  system’s  outputs.  Usability  testing  tests  user  satisfaction 
with  system  interface.  Once  the  system  is  developed  and  tested,  it  is  time  for 
implementation  (Gordon  &  Gordon,  1999). 

The  implementation  phase  activates  the  new  system.  At  this  point  a  pilot  test  may 
be  accomplished.  This  test  allows  a  small  portion  of  system  users  to  use  the  system  as 
they  would  in  reality.  This  allows  the  developers  to  phase  in  the  new  system  while 
working  out  any  problems.  The  developers  can  also  choose  to  perform  a  direct  cutover 
where  the  new  system  replaces  the  old  system  overnight.  Another  important  facet  of  this 
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phase  is  training  the  system’s  users.  Once  the  system  is  in  use,  maintenance  is  required 
to  keep  the  system  functioning  as  needed  (Gordon  &  Gordon,  1999). 

This  final  phase,  the  maintenance  phase,  serves  to  continuously  fix  any  system 
errors  or  “bugs”.  In  this  phase  the  system  can  also  be  modified  with  new  features  to 
improve  performance.  These  new  features  could  even  be  beyond  the  original  needs 
assessment.  This  phase  could  seemingly  last  the  lifetime  of  the  system  (Gordon  & 
Gordon,  1999). 

The  phases  in  each  of  these  models,  while  different,  provide  similar  overall 
guidance  to  system  development.  However  none  of  these  models  provide  a  methodology 
to  accomplish  the  appropriate  phases.  To  move  through  the  phases  of  the  ISDLC, 
developers  need  an  approach.  Gordon  and  Gordon  (1999)  posited  the  development 
pathways  (i.e.,  the  waterfall  approach,  the  spiral  approach,  and  prototyping).  Peters 
(1987)  also  described  the  waterfall  approach  and  prototyping. 

Information  Systems  Development  Life  Cycle  Approaches 

The  traditional  approach  to  the  ISDLC  contains  numerous  phases  and  provides  a 
framework  for  system  development.  As  stated  before,  authors  may  posit  different  phases 
but  they  all  contain  similar  components.  The  first  two  examples  of  the  ISDLC  above  use 
the  traditional  approach  which  requires  that  the  steps  of  each  phase  must  be  completed 
before  moving  on  to  the  next  phase  (Brookes  et  ah,  1982;  Harpool  et  ah,  1987).  These 
methods  also  do  not  provide  a  detailed  representation  of  the  system  or  a  methodology  to 
system  development.  They  provide  a  guideline  of  the  required  steps  but  no  guidance  as 
to  how  to  complete  those  steps  (Aktas,  1987). 
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Furthermore,  the  traditional  approach  assumes  that  new  systems  progress 
smoothly  and  sequentially  through  the  phases  when,  in  fact,  all  the  steps  in  these  phases 
are  not  necessarily  completed  in  sequence  (Gordon  &  Gordon,  1999).  Therefore, 
problems  with  this  traditional  approach  to  the  ISDLC  have  led  to  other  approaches 
(Aktas,  1987).  These  approaches  work  in  conjunction  with  the  ISDLC  and  provide 
guidance  for  system  developers  to  move  through  the  life  cycle.  Here,  for  example,  are 
some  of  the  approaches. 

The  waterfall  approach.  The  waterfall  approach  follows  the  ISDLC  in  sequence 
and,  in  some  ways,  is  similar  to  the  traditional  approach.  The  approved  results  or  outputs 
of  one  phase  are  fed  into  the  next  phase  but  the  waterfall  approach  allows  some 
interaction  between  the  phases  in  the  form  of  overlap  with  respect  to  time  (Gordon  & 
Gordon,  1999;  Peters,  1987).  A  graphical  model  is  found  in  Figure  1. 


Needs  Assessment 


Figure  1.  The  waterfall  model.  From  “Information  Systems:  A  management  approach” 
by  Gordon  and  Gordon  (1999). 

System  development,  in  this  case,  never  moves  backward.  This  approach  is  easy 
to  manage  and  the  sequence  allows  the  developer  to  set  deadlines  and  monitor  progress 
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but  there  are  disadvantages  as  well  (Gordon  &  Gordon,  1999).  As  with  the  traditional 
approach,  this  approach  is  highly  inflexible.  Changes  or  updates  to  the  system  must  be 
made  post  hoc,  which  can  increase  costs  and  cause  delays  (Gordon  &  Gordon,  1999). 
Furthermore,  no  portion  of  the  system  is  delivered  until  the  end  of  the  project.  These  are 
self-imposed  constraints  that  can  create  tension  between  the  developer  and  the  customer 
because  of  changing  customer  expectations.  By  the  time  the  system  is  complete  and 
available,  the  customer  may  want  or  need  different  system  requirements  (Gordon  & 
Gordon,  1999;  Peters,  1987). 

Prototyping.  This  approach  is  used  extensively  in  engineering  and  provides  a 
good  methodology  for  information  systems  as  well  (Peters,  1987).  Prototyping  focuses 
on  user  interface  and  allows  the  user  to  view  progress  early  and  often  (Gordon  &  Gordon, 
1999;  Peters,  1987).  Users  meet  with  developers  periodically  to  discuss  system 
requirements.  The  design  and  development  phases  are  repeated  until  users’  needs  are 
satisfied  (see  Figure  2).  Developers  create  a  prototype  of  the  system  based  on  the 
requirements.  Dummy  data  is  sometimes  entered  to  show  the  user  system  function.  This 
approach  is  excellent  for  use  with  small  to  medium  systems  (Gordon  &  Gordon,  1999). 
Just  as  with  the  other  approaches,  there  are  advantages  and  disadvantages  to  prototyping. 

Prototyping  provides  plenty  of  advantages.  First  of  all,  the  amount  of  time 
between  analysis  and  implementation  is  decreased.  The  new  system  addresses  user  needs 
and  shows  the  benefits  of  the  system  before  cost  and  effort  get  out  of  control.  It  allows 
users  to  articulate  what  they  do  not  like  about  the  system.  This,  in  turn,  leads  to  system 
refinement  and  improvement  (Gordon  &  Gordon,  1999;  Peters,  1987). 
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Figure  2.  The  prototyping  model.  From  “Information  Systems:  A  management 
approach”  by  Gordon  and  Gordon  (1999). 

Prototyping  is  not  without  its  disadvantages.  Users  may  raise  their  expectations 
of  the  system  beyond  budgets.  For  instance,  users  may  continue  to  ask  for  more  features, 
causing  runaway  development,  which  leads  to  higher  costs  (Gordon  &  Gordon,  1999; 
Peters,  1987).  Therefore,  cost  savings  using  this  type  of  system  are  not  guaranteed. 
Finally,  full  system  functionality  may  be  delayed  until  the  end  of  the  project  (Gordon  & 
Gordon,  1999). 

In  both  cases,  these  approaches  provide  more  flexibility  than  the  traditional 
approach.  Furthermore,  each  has  its  advantages  and  disadvantages  and  system 
developers  can  choose  the  correct  method  for  their  particular  needs.  Developers  can 
choose  an  approach  based  on  their  given  circumstances  (Peters,  1987).  Each  of  these 
methods  has  seen  success.  These  approaches  define  how  to  move  through  the  ISDLC  but 
do  not  provide  the  tools  to  develop  a  system’s  structure.  A  structured  approach  to 
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analysis  and  design  can  provide  the  necessary  tools  for  system  development  (Aktas, 
1987). 

Structured  Approach  to  Analysis  and  Design 

Aktas  (1987)  presented  a  structured  approach  to  the  ISDLC.  The  structured 
approach  provides  the  system  developer  with  the  necessary  tools  and  techniques  to  use  in 
conjunction  with  the  ISDLC  to  successfully  produce  an  information  system.  To  present 
this  structured  approach,  Aktas  (1987)  proposed  a  system  life  cycle  and  highlighted  the 
tools  available  for  system  developers. 

Aktas’  (1987)  performed  a  content  analysis  of  four  ISDLCs.  He  then  noted  their 
similarities  and  found  that  the  process  was  very  similar  to  those  of  engineering  systems. 
The  phases  were:  planning,  analysis,  physical  design,  implementation  or  construction, 
and  maintenance. 

During  the  planning  phase  the  developer  sends  a  request  for  a  system  study, 
conducts  the  initial  investigation,  and  conducts  a  feasibility  study.  The  developer  gathers 
information  about  the  problem  and  requirements  and  determines  solutions  to  the  problem. 
Furthermore,  he  or  she  determines  constraints  to  system  development. 

In  the  analysis  phase,  the  problem  is  redefined.  The  developer  then  works  to 
understand  the  existing  system  (if  applicable)  or  requirements  for  the  new  system.  In 
doing  this,  he  or  she  detennines  user  requirements  for  the  new  system.  During  the  final 
step  of  the  analysis  phase,  the  developer  builds  a  conceptual  model  of  the  recommended 
system  based  on  user  requirements.  There  are  many  tools  available  for  the  developer  to 
build  the  conceptual  model  (e.g.,  hierarchy  charts  and  data  flow  diagrams). 
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The  third  phase,  the  design  phase,  produces  a  system  design  and  detailed  design. 
The  developer  takes  the  conceptual  design  from  the  analysis  phase  and  converts  it  into 
actual  specifications  for  the  system.  Once  the  specifications  are  set,  the  developer  builds, 
tests,  and  installs  the  system.  This  is  known  as  the  implementation  phase.  Also  during 
this  phase,  he  or  she  operates  the  system  looking  for  refinement  opportunities.  The  last 
step  of  the  implementation  phase  focuses  on  the  post-implementation  review.  Post¬ 
implementation  review  consists  of  determining  whether  or  not  the  system  is  meeting  user 
requirements  and  making  the  necessary  updates  or  changes  to  finalize  the  system. 

The  final  phase  of  the  life  cycle  is  the  maintenance  phase.  Here  the  developer 
performs  enhancements  to  the  system  as  required  by  the  user.  This  stage  continues  until 
a  new  system  is  required.  See  Figure  3  for  the  life  cycle  model. 

Unlike  with  the  traditional  approach  to  the  life  cycle  model,  the  system  developer 
is  free  to  move  back  and  forth  between  phases.  This  allows  for  customer  feedback  during 
each  phase,  ensuring  fewer  post  hoc  changes  and  refinements.  Furthermore,  portions  of 
the  system  can  be  delivered  before  the  end  of  the  project.  Each  of  these  advantages  can 
reduce  costs,  delays,  and  tension  between  the  developer  and  the  customer. 

As  stated  before,  the  life  cycle  only  provides  vague  guidelines  for  system 
development.  The  system  developer  needs  tools  and  techniques  to  ensure  system 
development  is  successful  (Aktas,  1987).  Hierarchy  charts  and  data  flow  diagrams  are 
examples  of  these  tools.  Multi-dimensional  data  models  represent  the  underlying  data 
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Figure  3.  Five  phases  of  the  information  systems  life  cycle.  From  “Structured  Analysis 
and  Design  of  Information  Systems”  by  Aktas  (1987). 


Hierarchy  charts.  A  hierarchy  chart  (or  function  chart)  shows  the  hierarchical 
relationships  of  the  system’s  components.  This  type  of  chart  is  similar  to  an 
organizational  chart  where  each  lower  level  is  a  subset  of  the  level  above  it.  Major  loops 
and  decisions  are  not  shown  in  a  hierarchy  chart.  A  hierarchy  diagram  shows  the  overall 
structure  of  the  system  and  its  components  or  functions.  Descriptions  of  the  components 
and  functions  are  included  with  the  hierarchy  chart  (Aktas,  1987).  Hierarchy  charts 
provide  a  concise  view  of  the  overall  system  and  its  function.  See  Figure  4  for  an 
example  of  a  hierarchy  chart. 


Figure  4.  The  structure  of  a  hierarchy  chart. 
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Data  flow  diagrams.  Data  flow  diagrams  (DFDs)  are  used  to  model  the  flow  of 
data  between  processes  and  data  stores  (Gordon  &  Gordon,  1999).  They  provide  a 
logical  map  of  a  proposed  or  existing  system  and  are  used  during  the  analysis  phase  of 
the  system  development  life  cycle  to  build  the  architectural  (conceptual)  design  of  the 
system. 

DFDs  have  many  capabilities.  They  are  good  for  partitioning  systems  into 
subsystems  and  showing  how  data  flows  throughout  the  system.  They  identify  data 
stores  and  in-flowing  and  out-flowing  data.  External  entities,  or  sources  of  the  system, 
are  also  represented  (Aktas,  1987).  Various  symbols  are  used  to  define  data  flow, 
processes,  external  entities,  and  data  stores.  These  symbols  are  relatively  universal  and 
Microsoft  Visio©  provides  the  capability  to  draw  DFDs.  The  symbols  and  their 
nomenclature  can  be  found  in  Figure  5. 
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Figure  5.  Data  flow  diagram  symbols. 


DFDs  can  be  used  to  show  different  levels  of  the  system.  The  context  diagram 
shows  the  system  boundaries.  A  zero-level  diagram  provides  an  overview  of  the  system, 
its  major  processes  and  data  flows.  Each  major  process  is  decomposed  and  shown  on  a 
lower  level  DFD.  These  are  referred  to  as  Level  1  diagrams.  If  required,  processes  from 
Level  1  diagrams  can  be  further  decomposed  and  presented  in  Level  2  diagrams.  This 
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process  is  repeated  until  the  proper  level  of  detail  is  accomplished.  Any  one  diagram, 
however,  should  be  limited  to  seven  (plus  or  minus  two)  processes.  Data  flows  can  also 
be  decomposed  and  represented  on  lower  level  diagrams.  See  Figure  6  for  an  example  of 
a  data  flow  diagram. 
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Figure  6.  Example  of  data  flow  diagram. 


In  this  example  the  source  inputs  information.  The  source  could  be  any  external 
entity  such  as  an  employee  or  customer.  The  system  processes  the  information.  During 
the  process  data  is  sent  to  and  retrieved  from  the  data  store,  usually  a  database.  The 
system  then  provides  an  output  of  information  to  the  sink.  The  sink  is  also  an  external 
entity  and  could  be  the  same  employee  or  customer  that  input  the  original  information. 

While  DFDs  are  useful  for  system  representation,  they  also  have  their  limitations. 
Aktas  (1987)  posits  that  DFDs  do  not  show  the  composition  of  the  data,  data  access 
requirements  of  data  stores,  decisions,  loops,  calculations,  or  quantities  for  data  and/or 
processes.  Despite  these  limitations,  Aktas  (1987)  presents  some  reasons  to  use  DFDs. 
DFDs  can  help  analysts  summarize  the  system  and  its  key  components  and  functions. 
Analysts  can  also  gain  an  understanding  of  the  relationships  within  the  system.  DFDs  are 
an  excellent  communication  tool.  Users  and  analysts  can  use  them  to  discuss  system 
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requirements  and  development.  Finally,  by  setting  up  the  logical  framework  with  a  DFD, 
the  system  developer  can  detennine  the  physical  characteristics  of  the  system  and  the 
feasibility  of  its  implementation. 

Multi-dimensional  data  model.  While  hierarchy  charts  and  DFDs  provide  a 
representation  of  the  overall  system  structure,  multi-dimensional  data  models  can  be  used 
to  represent  the  underlying  data  structure.  These  models  are  powerful  tools  that  provide 
views  of  large  relational  databases  from  multiple  dimensions,  allowing  managers  to  view 
data  in  different  ways  (Gordon  &  Gordon,  1999;  Laudon  &  Laudon,  2002).  The  multi¬ 
dimensional  view  provides  analysis  through  flexible  access  to  data  by  hiding  the  complex 
query  syntax  from  the  user  and  providing  a  quick  query  response  to  summarized,  high- 
level  data  (Connolly  &  Begg,  2002). 

Multi-dimensional  databases  are  compact  and  easy  to  understand  and  are  optimal 
for  visualizing  and  manipulating  data  with  many  inter-relationships.  Furthermore,  they 
minimize  data  storage  requirements.  To  optimize  database  efficiency,  dimensions  are 
created  and  pre-aggregated  into  a  hierarchical  structure.  For  instance,  if  tracking 
timeframes,  dimensions  are  created  for  years,  quarters,  months,  days,  and  hours.  Data  in 
this  format  can  be  consolidated,  drilled  down,  or  sliced  and  diced.  In  other  words,  the 
data  can  be  viewed  to  the  level  of  detail  required  by  the  user  or  from  different  viewpoints 
(Connolly  &  Begg,  2002). 

Multi-dimensional  structures  store  data  and  relationships  and  can  be  represented 
by  models  that  are  intuitively  analytical  and  easy-to-use.  Data  can  be  structured  as  a 
three-dimensional  cube,  which  represents  cells  of  data  in  an  array.  Data  can  be  visualized 
as  cubes  with  each  side  representing  a  dimension.  Laudon  and  Laudon  (2002)  provided 
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an  example.  A  company  supplies  four  different  products  across  three  regions.  Managers 
may  want  information  about  actual  and  projected  sales  for  each  product  across  the  three 
regions,  requiring  a  multi-dimensional  analysis.  See  Figure  7  for  an  example  of  the  data 
model  showing  product  versus  region. 


Product 


Region 


Figure  7.  Example  of  a  multi-dimensional  data  model.  From  “Management  information 
systems:  managing  the  digital  firm”  by  Laudon  and  Laudon  (2002). 


The  ability  to  rotate  the  cube  to  get  different  view  of  the  data  is  the  key  to  the 
multi-dimensional  model.  This  was  referred  to  above  as  slicing  and  dicing  the  data.  For 
instance,  if  the  cube  is  rotated  90  degrees,  the  face  shows  product  versus  actual  and 
projected  sales.  Many  views  are  possible,  allowing  managers  to  gain  a  good  perspective 
of  product  sales  across  regions  (Laudon  &  Laudon,  2002). 

Information  System  Usability 


A  well-designed  infonnation  system  should  be  usable.  It  is  necessary  to  assess 
usability  to  draw  conclusions  about  your  system.  Commonly,  usability  assessments  are 
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conducted  by  asking  participants  to  use  the  system  of  interest  in  specific  scenarios.  If  the 
goal  of  the  system  is  user  satisfaction,  subjective  usability  measures  can  be  used. 
Subjective  usability  measures  assess  attitudes  toward  system  attributes  such  as  ease  of 
use  and  quality  of  interface  (Lewis,  1995).  Some  posit  that  attitudes  towards  these 
attributes  affect  user  satisfaction  and  acceptance  of  the  system  and  whether  or  not  people 
will  continue  to  use  it  (Davis,  1989;  Lewis,  1995).  Therefore,  usability  is  an  important 
factor  to  consider  during  system  development. 

Usability  is  a  multi-dimensional  construct.  It  can  be  defined  and  measured  using 
a  multitude  of  variables  or  constructs.  While  these  constructs  may  be  presented  under 
different  nomenclature,  those  introduced  by  Davis  (1989)  and  Lewis  (1995)  presented 
constructs  that  they  feel  measure  system  usability.  Davis  (1989)  presented  perceived 
ease  of  use  and  usefulness  in  his  technology  acceptance  model.  He  posited  that  these  are 
responsible  for  system  acceptance  behaviors.  Lewis  (1995)  presented  overall  system 
usability  as  a  combination  of  system  usefulness,  information  quality,  and  interface 
quality.  These  constructs  can  be  used  to  measure  a  system’s  overall  usability. 

Davis  (1995)  defined  perceived  ease  of  use  as  “the  degree  to  which  the 
prospective  user  expects  the  target  system  to  be  free  of  effort.”  A  system  that  is  flexible, 
easy  to  learn,  and  easy  to  become  skillful  using  will  most  likely  be  considered  easy  to 
use.  Perceived  usefulness,  as  defined  by  Davis  (1995),  is  “the  prospective  user’s 
subjective  probability  that  using  a  specific  application  system  will  increase  his  or  her  job 
performance  within  an  organizational  context.”  A  system  that  allows  the  user  to  improve 
his  or  her  perfonnance,  productivity,  and  effectiveness  on  the  job  will  most  likely  be 


considered  useful. 
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In  order  to  measure  user  satisfaction  with  overall  system  usability,  Lewis  (1995) 
posited  three  usability  constructs:  system  usefulness,  information  quality,  and  interface 
quality.  While  there  are  no  formal  definitions  of  these  constructs  in  the  literature,  they 
each  focus  on  different  and  important  areas  of  system  usability.  System  usefulness 
consolidated  the  Davis  (1995)  constructs  perceived  ease  of  use  and  usefulness. 
Information  quality  dealt  with  how  the  system  presented  infonnation.  A  system  with 
good  infonnation  quality  presented  information  in  a  clear  and  understandable  manner, 
allowed  the  user  to  find  information  easily,  and  provided  help  when  the  user  made  a 
mistake  or  had  a  problem.  Interface  quality  focused  on  the  interaction  the  user  had  with 
the  system.  A  user  expected  a  system  with  good  interface  quality  to  have  all  required 
functions  and  capabilities.  Those  functions  and  capabilities  also  provided  a  pleasant 
interaction  with  the  system.  Screen  graphics  and  icons  were  examples  of  interface  items 
that  affected  interface  quality. 

Despite  the  fact  that  usability  is  a  difficult  concept  to  define,  Davis  (1989)  and 
Lewis  (1995)  presented  constructs  to  measure  it.  These  measures  provide  valuable 
information  to  managers  who  are  looking  to  measure  user  satisfaction  of  a  system’s 
usability  and  its  affect  on  the  acceptance  of  that  system. 

Chapter  Summary 

This  chapter  provided  an  overview  of  developmental  feedback  and  the  feedback 
process  to  behavior  change,  presented  the  theory  and  development  of  the  feedback 
instruments,  and  highlighted  the  advantages  and  limitations  of  web-based  surveys.  The 
chapter  shifted  gears  and  focused  on  system  development,  specifically  the  systems 
development  life  cycle,  the  structured  approach  to  analysis  and  design,  and  the  tools 
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necessary  for  successful  analysis  and  design.  Finally,  the  chapter  highlighted  system 
usability,  some  constructs  that  can  be  used  to  define  it,  and  its  importance  to  user 
satisfaction  of  a  system’s  usability  and  system  acceptance. 
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III.  Methodology 

As  stated  in  Chapter  I,  the  objective  of  this  research  were  to  determine  the 
theoretical  framework  of  an  effective  developmental  feedback  program  and  to  develop 
and  implement  a  worthy  web-based  program  for  Aeronautical  Systems  Center,  Air  Force 
Security  Assistance  Center,  and  Air  Force  Materiel  Command.  A  structured  approach  to 
the  information  systems  life  cycle  was  used  to  design  the  web-based  developmental 
feedback  system.  The  methodology  used  for  this  research  completed  four  phases  of  the 
five-phase  information  systems  life  cycle  depicted  in  Table  3.  These  were  the  planning, 
analysis,  physical  design,  and  implementation  phases. 

The  system  development  phases  were  tailored  from  Aktas  (1987)  to  meet  the 
requirements  of  this  study.  It  is  important  to  note  that  these  phases  and  their  steps  were 
not  necessarily  followed  sequentially;  they  were  accomplished  iteratively.  For  instance, 
work  on  some  phases  or  steps  required  returning  to  the  previous  phase  or  step  and 
revising  what  had  been  accomplished.  The  results  for  the  completed  phases  and  their 
steps  are  found  in  Chapter  IV. 


Phase  1.  Planning 

The  purpose  of  the  planning  phase  was  to  gather  information  about  the  research 
problem  and  general  system  requirements.  It  was  also  used  to  set  the  criteria  for  a 
solution.  An  initial  investigation  and  feasibility  study  were  conducted  to  meet  the 
requirements  of  this  phase. 


Table  3 

Structured  Approach  to  Systems  Development 
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_ Phase _ Steps _ 

1 .  Planning  1.1.  Initial  investigation 

1.2.  Feasibility  study 

2.  Analysis  2.1.  Redefine  the  problem 

2.2.  Detennine  system  requirements 

2.3.  Architectural  design 

3.  Physical  design  3.1.  Actual  system  design 

4.  Implementation  4.1.  System  building 

4.2.  Testing 

4.3.  Installation 

4.4.  Operations 

4.5.  Post-implementation  review 

5.  Maintenance  5.1.  Maintenance  and  enhancements 

Note.  Adapted  from  “Structured  Analysis  and  Design  of  Information  Systems,”  by  Aktas 
(1987). 

The  feedback  program  developed  by  Patton  (2002)  had  originally  met  all 
customer  requirements  and  provided  a  good  framework  for  the  web-based  program.  It 
was  known  that  the  program  needed  to  be  automated  to  exploit  the  advantages  a  web- 
based  program  had  over  a  paper-based  program,  mainly  time  savings,  cost  savings,  and 
geographical  reach.  Thus,  the  purpose  of  the  initial  investigation  was  to  reveal  general 
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requirements  and  features  of  an  effective  automated  program  that  would  still  meet 
customer  requirements. 

After  general  requirements  were  detennined,  the  researcher  moved  on  to  the 
feasibility  study.  The  main  focus  of  the  feasibility  study  was  to  answer  the  question, 
could  a  web-based  system  with  the  general  requirements  and  features  noted  in  the  initial 
investigation  be  developed?  To  answer  this  question,  the  criteria  for  system  development 
were  identified  and  are  presented  in  Chapter  IV.  If  a  system  that  met  the  proposed 
criteria  could  be  developed,  the  researcher  would  deem  that  system  development  was 
feasible  and  would  move  to  the  analysis  phase  of  the  life  cycle. 

Phase  2.  Analysis 

The  analysis  phase  consisted  of  redefining  the  problem,  determining  system 
requirements,  and  building  an  architectural  design. 

Redefining  the  Problem 

It  was  necessary  to  design  a  web-based  program  that  was  easy  to  use  and  would 
provide  the  necessary  tools  for  a  leader  to  move  through  the  feedback  process  and 
improve  his  or  her  leadership  behaviors.  This  system  needed  to  provide  savings  in  time 
and  cost  over  a  traditional  paper-based  program.  Finally,  a  large  population  should  have 
access  to  the  system  through  the  website. 

Determining  System  Requirements 

System  requirements  were  developed  during  this  phase  and  defined  what  the 
system  needed  to  do.  The  needs  were  synthesized  into  a  baseline  of  system  level 
requirements  that  could  be  translated  into  an  architectural  design.  The  first  investigative 
question  was  used  to  establish  the  requirements  baseline. 
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Investigative  question  1.  What  system  capabilities  are  dictated  by  developmental 
feedback  requirements  that  will  allow  the  leader  to  easily  move  through  the  feedback 
process  to  behavior  change? 

System  requirements  were  extracted  from  the  literature  review,  the  Patton  (2002) 
study,  and  a  brainstonning  exercise  and  were  used  to  develop  the  requirements  baseline. 
This  baseline  represented  the  theoretical  framework  of  an  effective  developmental 
feedback  program  and  defined  system  level  capabilities  that  would  allow  a  leader  to 
easily  move  through  the  feedback  process  to  improve  his  or  her  leadership  behaviors. 

The  developmental  feedback  functions  (i.e.,  feedback  collecting,  results  reporting, 
and  action  planning)  from  the  literature  review  were  defined  as  the  capabilities  an 
effective  developmental  feedback  program  should  have.  An  analysis  of  the  Patton  (2002) 
study  was  used  to  obtain  specific  examples  of  the  required  functions  (i.e.,  feedback 
instruments,  reports,  action  planning  guide).  Finally,  the  researcher  brainstormed 
possible  system  features  that  would  make  the  system  self-sufficient  and  easier  for  a 
leader  to  use. 

Architectural  Design 

Once  the  requirements  baseline  was  established,  the  architectural  design  was 
developed  to  represent  the  overall  system  structure  and  its  processes.  The  architectural 
design  was  a  user  orientation  of  the  system  design.  Investigative  question  2  was  used  to 
develop  an  architectural  design  of  the  web-based  developmental  feedback  system. 

Investigative  question  2.  How  do  the  requirements  affect  system  architecture? 

System  architecture  was  developed  applying  a  structured  approach  to  the 
requirements  baseline.  A  hierarchy  chart  was  developed  to  represent  overall  system 
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structure.  A  series  of  data  flow  diagrams  was  used  to  shoe  different  levels  of  the  system, 
their  functions,  and  their  the  flow  of  data  between  their  processes  and  data  stores. 

Finally,  three-dimensional  data  models  were  developed  to  represent  the  underlying  data 
structure  of  the  system.  Once  this  conceptual  design  once  established,  it  was  possible  to 
develop  actual  system  specifications. 

Phase  3.  Physical  design 

During  the  physical  design  phase,  the  architectural  design  was  translated  into  an 
actual  system  design.  Investigative  question  3  was  used  to  develop  an  actual  system 
design. 

Investigative  question  3.  How  does  the  system  architecture  affect  the  actual 
system  design? 

While  the  architectural  design  provided  functional  requirements  and 
specifications,  the  system  design  provided  actual  system  specifications.  In  other  words, 
specifications  of  the  website,  feedback  instrument,  feedback  reports,  and  on-line 
workbook  were  developed  during  this  phase.  These  specifications  were  derived  from  the 
requirements  baseline  and  system  architecture  and  resulted  in  a  series  of  templates. 

These  templates  provided  basic  specifications  for  content  of  each  web  page,  e-mail, 
report,  and  workbook  page  for  the  entire  system.  Only  the  requirements  for  each  page 
were  provided  on  the  template.  The  system  programmer  determined  graphics  design. 
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Phase  4.  Implementation 

The  implementation  phase  consisted  of  system  building,  testing,  installation, 
operations,  and  post-implementation  review.  Investigative  question  4  was  used  to 
construct  the  actual  system. 

Investigative  question  4.  How  do  you  translate  the  architectural  and  physical 
designs  into  a  working  information  system? 

System  Building 

Once  the  system  specifications  were  developed,  an  experienced  programmer  was 
required  to  build  the  actual  system.  Two  contractors  that  worked  for  the  graduate  school 
programmed  the  system  from  the  architectural  and  physical  designs.  They  used  the 
architectural  design  to  detennine  the  overall  “picture”  of  the  system  and  its  requirements. 
The  physical  design  templates  were  used  to  construct  the  actual  web  pages,  database,  and 
reports. 

Testing 

Once  the  system  was  developed,  it  was  tested  for  functionality  and  ease  of  use. 
Thirty-seven  students  and  faculty  members  from  the  graduate  school  were  recruited  to 
test  the  system.  Participants  were  given  an  overview  of  the  system,  its  function,  and  the 
link  to  the  system  via  e-mail.  No  system  navigation  instructions  were  given  to  the 
participants. 

Students  were  given  scenarios  to  complete.  For  instance,  one  student  was 
assigned  as  a  leader  while  others  were  assigned  as  his  assessors  (i.e.,  peers  and  team 
members).  The  “leader”  used  the  system  to  obtain  feedback  from  the  assessors.  The 
assessors  completed  the  LCI-Observer.  The  results  of  this  portion  of  the  test  were  used  to 
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determine  whether  or  not  the  system  was  functioning  as  designed.  For  instance,  the 
researcher  used  some  of  the  responses  to  test  the  scale  values  to  ensure  values  were  being 
properly  entered  into  the  database. 

Faculty  members  were  instructed  to  use  the  system  as  a  way  to  assess  their 
leadership  behaviors.  They  were  given  the  liberty  to  choose  their  own  assessors  and 
were  requested  to  use  the  system  equivalent  to  its  intended  purpose.  This  was  used  to 
view  system  function  in  a  real  world  scenario. 

All  participants  (students  and  faculty)  were  instructed  to  identify  any  bugs  in  the 
system  and  provided  feedback  on  its  ease  of  use.  To  measure  ease  of  use,  the  researcher 
added  four  items  to  the  end  of  the  LCI-Observer  (e.g.,  “I  found  this  web  survey  easy  to 
use”).  The  ease  of  use  scale  is  described  in  further  detail  later  in  this  chapter.  There  was 
an  open-ended  item  at  the  end  of  both  feedback  instruments  and  personnel  were 
encouraged  to  provide  comments  about  the  system.  Participants  were  also  encouraged  to 
provide  any  feedback  about  the  system  to  the  research  team  via  e-mail  or  in  person  if 
necessary. 

Installation 

Since  there  was  no  previous  version  of  the  web-based  developmental  feedback 
system,  the  current  system  was  installed  for  the  test  by  the  school’s  programmers.  To  do 
this,  the  programmers  activated  the  web  site  and  its  functions.  The  only  requirements 
installed  after  testing  were  those  items  changed  or  updated  as  a  result  of  the  system  test. 
Operations 

Once  the  system  was  tested  and  the  updates  were  installed,  the  system  was 
fielded.  The  system  was  originally  fielded  to  students  and  faculty  at  the  graduate  school. 
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It  was  also  advertised  to  graduate  students  at  another  school  in  the  area.  Potential 
participants  were  given  an  overview  of  the  system,  its  function,  and  the  link  to  the  system 
via  e-mail.  To  access  the  system,  leaders  clicked  on  the  link  provided  in  the  e-mail.  The 
leaders  were  then  presented  with  the  home  page  and  could  operate  the  system  as  desired. 
The  system  was  self-sufficient  and  required  no  researcher  actions  to  operate.  The  system 
did  have  contact  information  that  allowed  users  to  contact  the  researcher  with  questions, 
feedback,  or  problems.  All  questions  or  problems  were  resolved  as  necessary.  The 
researcher  retrieved  data  from  the  system  via  the  school’s  programmer.  The  programmer 
extracted  the  data  from  the  database  and  provided  a  spreadsheet  to  the  researcher. 
Post-implementation  Review 

Once  the  system  was  operational  it  was  necessary  to  obtain  information  on  the 
program  and  infonnation  system.  Particularly,  the  researcher  wanted  to  obtain  data  on 
the  perceived  utility  of  the  feedback  and  perceived  usability  of  the  system.  Investigative 
question  5  was  used  to  assess  these  variables. 

Investigative  question  5.  How  do  you  assess  the  perceived  utility  of  the  feedback 
and  usability  of  the  infonnation  system? 

To  measure  feedback  utility,  the  researcher  relied  on  comments  provided  by 
leaders  and  observers  at  the  end  of  the  surveys.  All  comments  were  analyzed  for 
perceptions  of  utility  from  the  leader  and  observer  perspectives.  Furthermore,  the  LCI- 
Observer  contained  two  items  at  the  end  of  the  assessment  that  were  used  to  measure 
utility  from  the  observer  perspective.  An  analysis  of  the  descriptive  statistics  from  the 
two  items  combined  with  the  analysis  of  the  comments  was  used  to  fonnulate 
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conclusions  about  the  usefulness  of  the  feedback.  The  researcher  felt  that  leaders  and 
observers  would  find  the  feedback  useful  for  leadership  development. 

To  measure  system  usability,  any  comments  provided  by  leaders  and  observers 
that  related  to  ease-of-use  issues  were  noted.  Furthermore,  the  observer  assessment 
contained  four  items  that  were  used  to  measure  perceived  usability  from  the  observer 
perspective.  An  analysis  of  the  comments  and  descriptive  statistics  was  used  to 
determine  perceived  ease  of  use.  The  researcher  felt  that  people  would  find  the  system 
easy  to  use. 

Perceived  utility.  Brett  and  Atwater  (2001)  posited  in  their  feedback  model  that 
prior  to  behavior  change  the  feedback  recipient  must  find  the  feedback  useful.  Useful 
feedback  (i.e.,  feedback  that  motivates  the  recipient  to  set  goals)  will  most  likely  lead  to 
goal  setting,  a  prerequisite  to  behavior  change  (Locke  &  Latham,  1990).  Therefore, 
perceived  usefulness  of  the  feedback  is  a  necessary  requirement  for  goal  setting. 

Leaders  and  observers  provided  comments  as  to  the  usefulness  of  the  feedback 
provided  by  the  system.  Comments  from  leaders  and  observers  were  analyzed  to 
determine  perceived  utility  (i.e.,  usefulness)  of  the  feedback.  Furthermore,  observers 
rated  their  perceptions  of  the  utility  of  the  feedback  they  provided.  The  utility  items  used 
a  7-point  Likert  response  scale  ranging  from  l=Strongly  disagree  to  7=Strongly  agree. 
The  two  items  were,  “I  feel  I  have  worked  long  enough  with  this  person  to  adequately 
assess  his/her  leadership  behaviors”  and  “I  believe  the  information  collected  by  this 
survey  can  help  leaders  become  more  effective”. 

Perceived  usability.  Observers  reported  the  degree  to  which  they  felt  the  system 
was  free  of  effort.  Items  were  adapted  from  Davis’s  (1989)  perceived  ease  of  use  scale 
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and  Lewis’s  (1995)  system  usefulness  scale.  Davis  described  perceived  ease  of  use  and 
perceived  usefulness  as  two  separate  constructs.  Lewis  described  a  construct  called 
system  usefulness,  which  was  a  combination  of  perceived  ease  of  use  and  usefulness. 
Perceived  usefulness  focused  on  improving  a  user’s  productivity  and  effectiveness  on  the 
job.  In  this  study,  since  the  feedback  program,  not  the  system  itself,  was  designed  to 
improve  a  leader’s  perfonnance,  the  usefulness  construct  was  not  relevant.  Therefore, 
the  ease  of  use  items  from  Lewis’s  system  usefulness  scale  were  combined  with  Davis’s 
perceived  ease  of  use  scale  to  form  the  scale  for  this  study.  There  was  some  overlap 
between  some  items  and  the  scale  for  this  study  included  four  items  (i.e.,  “The  survey 
questions  were  easy  to  understand”;  “Learning  to  use  this  web  survey  was  easy  for  me”; 
“I  found  it  easy  to  get  the  web  survey  to  do  what  I  wanted  it  to  do”,  “I  found  this  web 
survey  easy  to  use”).  The  ease  of  use  scale  had  a  reliability  coefficient  of  .80,  surpassing 
the  required  value  of  .70  (Huck  &  Connier,  1996)  and  providing  evidence  that  the  scale 
is  reliable. 


Phase  5.  Maintenance 

The  maintenance  phase  is  ongoing  and  the  results  were  beyond  the  scope  of  this 

study. 


Chapter  Summary 

This  chapter  provided  the  methodology  used  to  design  the  web-based 
developmental  feedback  program.  In  particular,  the  relevant  phases  and  steps  of  the 
information  system  development  life  cycle  for  this  study  were  discussed.  The  results  for 
each  of  the  phases  are  discussed  in  Chapter  IV. 
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IV.  Results  of  the  Life  Cycle  Phases 

This  chapter  contains  the  results  of  the  first  four  stages  of  the  information  system 
development  life  cycle  (i.e.,  planning,  analysis,  physical  design,  and  implementation). 
The  planning  phase  was  designed  to  conduct  the  initial  investigation  for  the  study.  Also 
during  this  phase,  the  researcher  completed  a  feasibility  study.  Once  it  was  determined 
that  the  system  was  feasible,  the  researcher  moved  to  the  analysis  phase  of  the  life  cycle. 
During  this  phase,  the  problem  was  redefined  and  the  system  requirements  baseline  and 
architectural  design  were  developed.  From  the  architectural  design  the  researcher  was 
able  to  develop  the  physical  design  of  the  system  and  complete  phase  three  of  the  life 
cycle.  Programmers  used  the  architectural  and  physical  designs  to  implement  and  install 
the  system.  A  limited  assessment  of  feedback  utility  and  system  usability  was  conducted 
at  the  end  of  phase  four. 


Phase  1.  Planning 

The  planning  phase  consisted  of  an  initial  investigation  and  a  feasibility  study. 
During  the  initial  investigation  the  researcher  gathered  information  about  the  research 
problem  and  general  system  requirements.  The  feasibility  study  set  the  criteria  for  a 
solution. 

As  stated  in  Chapter  III,  the  paper-based  feedback  program  developed  by  Patton 
(2002)  had  originally  met  all  customer  requirements  and  provided  a  good  framework  for 
the  web-based  program.  A  web-based  program  would  automate  the  functions  of  the 
paper-based  program  and  exploit  the  advantages  of  electronic  data  input  and  output  and 
the  Internet.  The  functions  of  the  paper-based  program  served  as  templates  to  possible 
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system  inputs,  outputs,  and  features.  The  feedback  instruments  would  provide  the  inputs 
to  the  system  and  the  feedback  reports  would  provide  the  outputs.  This  program  also  had 
an  action  plan  workbook  that  was  used  by  the  leaders  to  set  goals  based  on  their 
feedback.  Therefore  it  was  determined  web-based  program  could  be  modeled  after 
Patton’s  feedback  program. 

By  automating  the  feedback  instruments,  reports,  and  action  plan  workbook,  the 
web-based  system  would  require  certain  added  features  to  provide  a  similar  feedback 
process  as  the  paper-based  program.  These  are  discussed  in  more  detail  later  in  the 
chapter.  Now  that  the  general  requirements  for  the  system  were  established,  the 
researcher  needed  to  know  if  it  was  feasible  to  proceed  with  the  study. 

To  determine  the  study’s  feasibility,  the  researcher  developed  certain  criteria  the 
system  would  have  to  meet.  The  criteria  were  as  follows.  The  system  had  to  exploit  the 
advantages  of  a  web-based  program  over  a  paper-based  program.  In  other  words,  the 
system  had  to  produce  cost  savings,  time  savings,  and  achieve  a  broad  geographical 
reach.  The  system  had  to  be  self  sufficient  and  easy  to  use.  That  is,  a  leader  should  be 
able  to  move  through  the  feedback  process  with  little  or  no  outside  help.  The  system  had 
to  be  accessible  by  personnel  Air  Force-wide  and  should  be  compatible  with  the  majority 
of  Air  Force  computers  and  browsers.  Finally,  due  to  the  lack  of  funds,  the  system 
needed  to  be  developed  by  one  of  the  graduate  school’s  programmers. 

After  a  short  interview  with  a  programmer  from  the  graduate  school  it  was 
determined  that  all  the  criteria  could  be  met  and  accomplished  “in-house”.  The  web- 
based  system  would  achieve  electronic  data  input  and  output  and  would  save  the 
researcher  from  performing  these  functions  manually,  thus  saving  time.  The  system 
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would  be  available  on  the  World  Wide  Web  and  would  achieve  a  broad  geographical 
reach.  In  light  of  this,  personnel  Air  Force-wide  would  have  access  to  the  system. 
Furthermore,  the  software  used  to  develop  the  system  was  compatible  with  all  Air  Force 
computers  and  browsers.  The  researcher  deemed  that  the  study  was  feasible  and  moved 
to  the  analysis  phase  of  the  life  cycle. 

Phase  2.  Analysis 

This  section  highlights  the  results  of  the  analysis  phase  of  the  development  life 
cycle.  This  phase  consisted  of  redefining  the  problem,  determining  system  requirements, 
and  building  an  architectural  design. 

Redefining  the  Problem 

Given  that  the  system  design  was  feasible,  it  was  necessary  to  design  the  system 
to  provide  the  necessary  tools  for  a  leader  to  move  through  the  feedback  process  and 
improve  his  or  her  leadership  behaviors.  With  the  problem  redefined,  it  was  deemed  that 
the  study  would  still  provide  the  required  system,  it  was  then  necessary  to  detennine  what 
the  system  needed  to  do. 

Determining  System  Requirements 

The  purpose  of  this  step  was  to  answer  investigative  question  1  and  established  a 
requirements  baseline  for  the  system.  The  baseline  defined  what  the  system  needed  to 
do. 

Investigative  question  1.  What  system  capabilities  are  dictated  by  developmental 
feedback  requirements  that  will  move  the  leader  through  the  feedback  process  to  behavior 
change? 
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The  requirements  were  extracted  from  the  literature  review,  the  Patton  (2002) 
study,  and  a  brainstorming  exercise.  The  developmental  feedback  functions  (i.e., 
feedback  collecting,  results  reporting,  and  action  planning)  from  the  literature  review 
were  defined  as  the  capabilities  an  effective  developmental  feedback  program  should 
have.  An  analysis  of  the  Patton  (2002)  study  was  used  to  obtain  specific  examples  of  the 
required  functions  (i.e.,  feedback  instrument,  reports,  action  planning  guide).  Finally,  the 
researcher  brainstormed  possible  system  features  that  would  make  the  system  self 
sufficient  and  easy  for  the  leader  to  use. 

Effective  developmental  feedback  is  designed  to  encourage  leaders  to  improve 
their  leadership  behaviors.  As  stated  above,  an  effective  program  should  provide  three 
functions  that  will  provide  the  necessary  tools  to  move  the  leader  through  the  feedback 
process  and  improve  his  or  her  leadership  behaviors.  These  three  functions  were 
identified  from  the  literature  as:  feedback  collecting,  results  reporting,  and  action 
planning. 

Feedback  collecting.  The  requirements  for  collecting  feedback  were  adapted 
from  the  literature  review  and  the  Patton  (2002)  study.  Leaders  and  their  raters  (also 
called  observers)  provide  developmental  feedback  through  a  feedback  instrument.  The 
main  requirement  for  collecting  feedback  is  to  ensure  that  the  leader  will  perceive  it  as 
accurate.  To  ensure  this,  both  the  instrument  and  raters  must  have  certain  characteristics. 

The  instruments  chosen  for  this  system  were  the  Leadership  Commitments 
Inventory  (LCI)  and  a  stakeholder  assessment.  The  LCI  has  not  been  validated  but  was 
based  on  two  valid  instruments  (i.e.,  LPI  and  UFI)  and  was  developed  using  statistical 
methods.  Lurthermore,  the  purpose  for  deploying  the  LCI  with  this  system  was  to  test  its 
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reliability  and  validity.  The  stakeholder  assessment  will  also  be  deployed  with  the 
system  to  provide  a  means  to  test  the  criterion  validity  of  the  LCI. 

The  LCI  focused  on  behaviors  that  are  important  to  the  sponsoring  organizations, 
letting  the  leader  and  raters  know  the  critical  behaviors  for  success.  The  instrument  was 
also  easily  translated  to  a  web-based  environment.  Given  these  characteristics,  if  the  LCI 
was  a  valid  instrument,  it  would  theoretically  provide  accurate  feedback. 

No  instrument  can  provide  accurate  feedback  without  raters.  Good 
developmental  feedback  programs  are  designed  for  the  leader  to  initially  perfonn  a  self- 
assessment  and  then  request  feedback  from  observers.  As  a  result,  two  versions  of  the 
LCI  (Self  and  Observer)  were  required.  The  system  needed  to  allow  the  leader  to 
perform  a  self-assessment  and  then  request  feedback  from  his  or  her  boss,  peers,  direct 
subordinates,  indirect  subordinates,  team  members,  and  customers.  To  inform  the  leader 
of  this  requirement,  the  system  needed  to  have  easily  accessible  guidance  for  obtaining 
feedback. 

To  increase  feedback  accuracy  and  interrater  reliability  and  to  ensure  rater 
anonymity,  the  leader  requires  that  between  three  and  five  raters  in  each  category  (except 
the  boss  category)  complete  the  LCI-Observer.  To  guarantee  confidentiality,  feedback 
must  be  processed  and  stored  by  the  system  and  its  database,  preferably  on  the  graduate 
school’s  web  server. 

Results  reporting.  Once  feedback  is  collected  it  must  be  presented  for  the  leader 
to  receive  and  understand  it.  Feedback  reports  provide  this  function.  Reports  and  their 
characteristics  are  keys  to  the  perceived  accuracy  and  usefulness  of  the  feedback. 
Therefore  this  system  needed  to  provide  reports  that  would  present  clear  and 


77 

understandable  feedback  infonnation.  Requirements  for  the  reports  were  adapted  from 
the  literature  review  and  Patton  study  (2002). 

To  provide  the  best  view  of  the  feedback,  three  types  of  reports  were  required  for 
this  system:  summary,  comparison,  and  stakeholder.  These  reports  should  be  adapted  to 
a  computer  screen  but  also  printable.  The  summary  report  needed  to  provide  a  graphical 
representation  summarizing  the  results  at  the  practice,  commitment,  and  item  level  for 
each  observer  category.  This  would  allow  the  leader  to  make  immediate  interpretations 
of  the  feedback  and  how  he  or  she  performed  in  a  specific  area. 

The  comparison  report  needed  to  provide  a  graphical  representation  allowing  the 
leader  to  compare  his  or  her  perceptions  to  the  perceptions  of  others.  The  leader  should 
be  able  to  compare  results  in  each  category  (e.g.,  self  to  peers,  self  to  direct  subordinate, 
etc.).  Since  the  LCI  was  designed  to  measure  leadership  at  the  commitment  level  these 
comparisons  needed  to  be  provided  at  the  commitment  level.  Furthermore,  to  make  it 
easy  for  the  leader  to  determine  where  discrepancies  occurred,  the  commitments  needed 
to  be  listed  in  order  of  greatest  discrepancy  to  least  discrepancy. 

The  stakeholder  report  needed  to  provide  a  graphical  representation  summarizing 
the  results  of  the  stakeholder  assessment.  This  would  allow  leaders  to  make  an 
immediate  interpretation  of  their  customer’s  satisfaction  with  the  performance  of  the  unit. 

Other  requirements  for  the  reports  function  dealt  with  particular  system  features. 
Since  the  system  was  to  be  self-sufficient,  the  reports  capability  needed  guidance  about 
the  three  types  of  reports  and  how  to  retrieve  and  read  the  reports.  Finally,  to  ensure 
anonymity  of  the  raters,  the  system  should  not  produce  results  for  a  specific  category 
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until  at  least  three  people  from  that  category  provide  feedback  (once  again  excluding  the 
boss  category). 

Action  planning.  Presenting  the  feedback  alone  will  not  result  in  behavior 
change.  Leaders  must  set  specific,  difficult  goals  before  behavior  change  can  take  place. 
Action  planning  provides  this  function.  This  system  needed  to  provide  a  facility  to  allow 
leaders  to  develop  action  plans  based  on  the  feedback  reports.  Since  it  was  not 
logistically  possible  to  provide  one-on-one  facilitated  or  workshop  development  sessions 
to  all  users,  the  system  needed  to  provide  a  development  workbook  or  self  study  guide. 
The  workbook  needed  to  assist  leaders  in  analyzing  reports,  extracting  their  strengths  and 
weaknesses,  and  developing  action  plans. 

Since  leaders  would  most  likely  not  be  familiar  with  action  planning,  the 
workbook  needed  to  provide  guidance  on  the  action  planning  process  (i.e.,  building, 
sharing,  and  implementing  the  action  plan).  To  guide  leaders  in  building  an  action  plan, 
the  workbook  needed  an  action  plan  template  and  a  sample  action  plan.  The  action  plan 
template  needed  to  give  the  leader  the  capability  to  set  improvement  goals  in  each 
leadership  area.  The  workbook  also  needed  to  provide  lists  of  suggested  actions  and 
readings  as  tools  to  guide  a  leader’s  improvement  in  a  specific  area.  The  suggested 
actions  could  be  used  as  goal  statements  in  the  action  plan.  Finally,  to  make  it  easy  for 
the  leader  to  navigate,  the  workbook  needed  to  separate  the  actions  and  readings  at  the 
practice  level. 

Other  system  requirements.  Since  the  system  was  to  be  self-sufficient,  there 
needed  to  be  some  added  features  that  would  allow  a  leader  to  securely  move  through  the 
entire  developmental  feedback  process  with  little  or  no  assistance.  To  get  the  leader 
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started,  the  system  needed  to  provide  some  background  on  developmental  feedback  and 
the  current  study.  It  also  needed  to  provide  definitions  of  the  leadership  behaviors  that 
are  highlighted  in  the  program.  Maybe  most  importantly,  it  needed  to  provide  an 
overview  of  how  to  navigate  the  feedback  process  using  the  web-based  system. 

In  order  to  provide  confidentiality  of  the  feedback  information,  the  system  needed 
to  allow  the  leader  to  have  secure  access  to  the  system  and  feedback  information.  In 
order  to  obtain  assistance  while  using  the  system,  there  needed  to  be  a  contact  e-mail  to 
allow  users  to  contact  the  research  team  or  technical  support  with  problems,  questions,  or 
concerns.  Once  users  completed  a  session,  they  needed  to  have  a  means  to  exit  the 
system. 

Once  the  requirements  for  the  system  were  developed,  they  were  integrated  into  a 
requirements  baseline.  The  requirements  baseline  (Appendix  A)  served  as  the  outline  for 
the  architectural  (conceptual)  design  of  the  system. 

Building  the  Architectural  Design 

System  architecture  was  developed  applying  a  structured  approach  to  the 
requirements  baseline.  The  architectural  design  was  developed  to  represent  the  overall 
system  structure  and  its  processes  and  was  a  user  orientation  of  the  system  design.  The 
design  was  developed  using  a  hierarchy  chart,  a  series  of  data  flow  diagrams,  and  three- 
dimensional  data  models.  The  completed  architectural  design  of  the  web-based 
developmental  feedback  system  answered  investigative  question  2. 

Investigative  question  2.  How  do  requirements  affect  system  architecture? 
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Hierarchy  chart.  A  hierarchy  chart  was  developed  to  represent  overall  system 
structure.  It  was  designed  to  show  how  the  required  functions  would  relate  to  each  other. 
The  hierarchy  chart  is  found  in  Figure  8. 


Figure  8.  Hierarchy  chart  of  web-based  developmental  feedback  system. 


The  overall  system  structure  showed  that  a  user  without  a  secure  account  could  access  the 
main  page  and  top-level  features.  This  was  designed  so  that  leaders  could  obtain 
information  about  the  developmental  feedback  program  and  the  current  study  without 
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creating  a  secure  account.  The  contact  and  exit  system  features  were  to  be  accessible 
throughout  the  system  so  that  the  user  could  request  help  or  exit  the  system  at  any  time. 

After  getting  an  overview,  leaders  decide  whether  or  not  they  want  to  participate 
in  the  program.  If  they  choose  to  participate  they  create  a  secure  account  and  enter  the 
system.  After  entering  the  system,  a  leader  gains  access  to  guidance  about  the 
developmental  feedback  process,  feedback  instruments,  feedback  reports,  and  the  action¬ 
planning  workbook.  These  features  will  be  discussed  in  more  detail  in  the  next  section. 
Finally,  the  chart  shows  that  if  they  have  already  created  an  account,  they  simply  enter 
the  system  and  access  the  functions  they  need.  Once  the  overall  system  structure  was 
established,  it  was  necessary  to  view  system  functions  and  their  processes. 

Data  flow  diagrams.  A  series  of  data  flow  diagrams  (DFDs)  was  used  to  show 
different  levels  of  the  system,  their  functions,  and  the  flow  of  data  between  their 
processes  and  data  stores.  Higher-level  diagrams  were  developed  first  and  were  then 
decomposed  into  lower  level  diagrams.  The  higher  level  diagrams  gave  an  overview  of 
the  processes  and  data  flows  while  the  lower  level  diagrams  provided  more  detail.  Four 
levels  of  diagrams  were  used:  context,  level  0,  level  1,  and  level  2. 

The  context  diagram  gave  a  simple  view  of  the  system’s  boundaries  and  presented 
the  purpose  of  the  system.  The  context  diagram  can  be  found  in  Figure  9.  This  diagram 
shows  that  the  leader  uses  the  system  to  move  through  the  developmental  feedback 
process.  After  moving  through  this  process  the  leader  should  have  enough  information  to 
develop  and  implement  an  action  plan.  The  system  will  not  produce  an  action  plan  for 
the  leader;  it  will  simply  provide  the  leader  with  the  necessary  tools  to  develop  one.  This 
diagram  also  shows  that  observers  and  stakeholders  provide  inputs  to  the  system  and 
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Figure  9.  Context  diagram. 

receive  outputs  from  the  system  as  well.  Observers  and  stakeholders  receive  requests  for 
feedback  from  the  system  and  input  feedback  to  the  system. 

The  level  0  DFD  provided  an  overview  of  the  entire  system,  its  major  processes 
and  data  flows.  The  level  0  DFD  can  be  found  in  Figure  10.  This  diagram,  while 
seemingly  complex,  shows  the  many  processes  a  leader  must  use  throughout  the  system. 
The  major  processes  were  defined  by  the  numbers  one  (1.)  through  six  (6.). 

The  diagram  shows  that  the  leader  can  request  background  information  (process 
1.)  or  create  an  account  (process  2.)  without  actually  entering  the  system.  They  simply 
must  have  access  to  the  web  page.  The  contact  and  exit  functions  were  not  deemed  major 
processes  and  therefore  were  not  included  in  the  level  0  diagram.  They  were  designed, 
however,  to  be  available  to  the  leader  at  any  time. 

Once  a  leader  creates  a  secure  account  he  or  she  can  access  the  main  system 
features  by  entering  the  system  (process  3.).  To  enter  the  system  the  leader  must  provide 
the  correct  user  name  and  password.  This  feature  ensures  the  leader’s  feedback  is 
confidential  and  only  accessible  by  the  leader.  Once  the  leader  successfully  enters  the 


Figure  10.  Level  0  DFD  (System  overview). 
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system,  he  or  she  now  has  access  to  the  feedback  instruments,  feedback  reports,  and  the 
action-planning  workbook.  The  leader  requests  feedback  via  the  feedback  instruments 
(process  4.).  The  raw  response  data  goes  directly  into  the  leader’s  account  file  in  the 
database.  When  the  leader  requests  a  report  the  raw  response  data  is  compiled  by  the 
system  and  presented  to  the  leader  in  a  report  format  (process  5.).  The  leader  can  also 
request  the  action-planning  workbook  that  provides  the  necessary  tools  for  developing 
and  executing  a  leadership  improvement  action  plan  (process  6.).  Now  that  the  system 
overview  was  established  it  was  necessary  to  decompose  each  of  the  major  processes. 

The  level  0  diagram  was  decomposed  in  to  six  level  1  diagrams,  one  for  each 
major  process.  The  first  level  1  diagram  shows  the  “get  background  information”  process 
( 1 .)  and  can  be  found  in  Figure  1 1 . 


Figure  11.  Level  1  DFD  for  process  1  (Background  information). 


The  leader  is  not  required  to  provide  any  infonnation  to  request  background  information. 
He  or  she  simply  requests  the  appropriate  link  and  the  infonnation  becomes  available  on 
the  screen.  There  are  two  types  of  background  information  that  can  be  accessed  by  the 
leader,  LCI  background  information  and  leadership  behavior  information.  These  are 
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presented  to  the  leader  in  the  form  of  LCI  background  and  leadership  practice  and 
commitment  definitions,  respectively. 

The  next  level  1  diagram  showed  the  “create  account”  process  (2.)  and  can  be 
found  in  Figure  12. 
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Figure  12.  Level  1  DFD  for  process  2  (Create  account). 

When  the  leader  decides  to  use  the  system,  he  or  she  requests  an  account.  They  are 
required  to  enter  specific  information  that  will  allow  the  system  to  identify  them  in  the 
future.  The  required  information  was  determined  to  be:  first  name,  last  name,  e-mail 
address,  user  identification,  and  password.  Once  this  information  is  entered,  the  create 
account  function  establishes  an  account  file  for  the  leader  and  inputs  the  leader’s  account 
information  into  his  or  her  database  file.  Once  the  infonnation  is  entered  into  the  account 
file,  the  leader  gets  a  confirmation  that  the  account  has  been  created.  The  account 
information  allows  the  leader  to  access  the  system  in  the  future  and  allows  the  system  to 
retrieve  a  leader’s  feedback  data  to  create  reports. 

Once  the  leader  has  created  an  account  they  have  access  to  the  main  system 
functions,  particularly  the  developmental  feedback  functions.  To  access  the  tools  they 
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must  enter  the  system  (process  3.)-  The  level  1  diagram  for  “enter  system”  can  be  found 
in  Figure  13. 
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Figure  13.  Level  1  DFD  for  process  3  (Enter  system). 


To  enter  the  system  the  leader  requests  system  login.  He  or  she  is  then  prompted  to  enter 
the  proper  user  identification  and  password.  This  information  will  be  checked  against  the 
account  information  in  the  leader’s  account  file.  If  the  user  identification  and  password 
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are  valid  the  system  provides  the  leader  with  certain  options  (i.e.,  guidance,  get  feedback, 
reports,  and  action  planning  tools). 

If  the  user  identification  or  password  is  incorrect  the  leader  receives  a  rejection 
message  and  is  prompted  to  reenter  the  correct  infonnation.  If  the  leader  forgets  the  user 
identification  or  password  he  or  she  can  request  the  information.  He  or  she  is  then 
prompted  to  enter  his  or  her  e-mail  address.  If  a  valid  email  address  is  entered  the  system 
accesses  the  leader  account  file  and  provides  the  requested  account  infonnation  to  the 
leader  via  an  e-mail  message.  The  leader  then  has  access  to  the  system. 

Some  leaders  may  need  to  change  the  information  in  their  account  files. 

Therefore,  the  system  has  a  feature  that  allows  the  leader  to  change  account  information. 
The  leader  requests  the  change  and  is  prompted  to  enter  the  new  information.  The  new 
information  is  updated  in  the  leader’s  account  file.  Once  the  information  is  updated,  the 
leader  receives  a  confirmation  message  that  information  has  been  changed. 

When  a  leader  enters  the  system,  he  or  she  is  given  a  number  of  options.  He  or 
she  can  obtain  guidance  about  the  use  of  the  system,  get  feedback,  get  reports,  or  take 
action.  He  or  she  is  able  to  contact  the  researchers  or  exit  the  system  when  necessary. 
Leaders  obtain  feedback  through  one  of  the  feedback  instruments.  This  is  accomplished 
through  the  “get  feedback”  process  (4.).  The  level  1  DFD  is  found  in  Figure  14.  Figure 
14  shows  that  a  leader  is  able  to  create  a  new  assessment  or  add  to  an  existing  one.  A 
leader  will  create  a  new  assessment  the  first  time  he  or  she  requests  feedback.  In  this 
case,  the  system  will  input  feedback  data  into  the  leader’s  account  file.  A  leader  will  also 
create  a  new  assessment  if  he  or  she  wishes  to  obtain  feedback  subsequent  to  his  or  her 
initial  assessment.  For  instance,  the  leader  would  like  to  see  if  the  perceptions  of  his 
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Figure  14.  Level  1  DFD  for  process  4  (Get  feedback). 


or  her  leadership  behaviors  have  improved.  In  this  case,  all  old  feedback  data  is 
transferred  into  another  database  and  the  new  feedback  data  takes  its  place. 

If  a  person  already  has  an  existing  assessment  and  would  like  to  obtain  feedback 
from  more  personnel,  he  or  she  can  add  to  the  existing  assessment.  In  this  case  the 
system  inputs  the  new  feedback  data  into  the  leader’s  account  file.  It  should  be  noted  that 
the  leader  should  only  be  able  to  add  to  an  existing  assessment  for  a  specific  time  period 
after  he  or  she  began  the  assessment.  This  allows  the  leader  to  obtain  timely  and  accurate 
feedback. 

The  processes  “create  a  new  assessment”  and  “add  to  an  existing  assessment” 
were  decomposed  into  level  2  diagrams.  The  level  2  diagram  for  “create  a  new 
assessment”  can  be  found  in  Figure  15.  When  a  leader  requests  a  new  assessment  he  or 
she  can  get  feedback  via  the  LCI-Self,  LCI-Observer,  and  stakeholder  assessment.  To 
obtain  a  self-assessment  the  leader  requests  the  LCI-Self.  The  system  provides  them  with 
the  questionnaire  in  web  form.  The  leader  then  completes  the  items  on  the  questionnaire. 


Figure  15.  Level  2  DFD  for  process  4. 1  (Create  new  assessment). 


00 

vo 


90 


As  the  leader  completes  the  assessment  the  data  is  fed  to  the  leader’s  account  file.  If  the 
assessment  is  not  the  leader’s  first  one,  all  old  self-assessment  data  is  transferred  to  a 
separate  database  and  the  new  data  replaces  it.  Once  the  leader  has  completed  the  self- 
assessment  he  or  she  receives  a  message  that  the  assessment  is  complete. 

Leaders  also  attain  feedback  data  from  observers  and  stakeholders  (customers). 
The  leader  must  request  observer  and  stakeholder  feedback  through  the  system.  Once  the 
feedback  is  requested,  the  system  provides  the  leader  with  a  feedback  request  e-mail 
template.  The  leader  enters  the  appropriate  e-mail  addresses  and  sends  the  message.  The 
observers  receive  the  e-mail  that  contains  a  link  that,  when  selected,  directs  them  to  the 
LCI-Observer.  At  this  time,  observers  are  required  to  enter  the  appropriate  code  from  the 
e-mail  message  to  identify  to  which  leader’s  account  the  feedback  should  be  stored. 

They  are  also  required  to  enter  their  relationship  to  the  leader.  As  they  complete  the 
questionnaire  the  data  is  input  to  the  appropriate  leader’s  account  file.  Once  again,  if  the 
new  assessment  is  not  the  leader’s  first  one,  all  old  self-assessment  data  is  transferred  to  a 
separate  database  and  the  new  data  entered.  Once  the  observer  has  completed  the 
assessment  he  or  she  receives  a  message  that  the  assessment  is  complete.  The  same 
process  is  used  for  the  stakeholder  assessment  except  that  the  e-mail  feedback  request 
and  stakeholder  questionnaire  are  worded  differently. 

The  “add  to  existing  assessment”  process  (4.2)  is  very  similar  to  the  “create  new 
assessment”  process  and  can  be  found  in  Figure  16.  The  only  difference  lies  in  the 
storage  of  the  data.  When  the  leader  adds  to  the  existing  assessment  the  data  is  input  to 
the  leader’s  account  file.  No  old  data  is  transferred  to  another  database. 
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Figure  16.  Level  2  DFD  for  process  4.2  (Add  to  existing  assessment). 
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Once  leaders  have  feedback  data  it  is  possible  for  them  to  retrieve  reports.  The 
level  1  diagram  for  the  “reports”  process  can  be  found  in  Figure  17. 


Figure  17.  Level  1  DFD  for  process  5  (Reports). 


The  leader  has  four  options  when  he  or  she  requests  reports.  He  or  she  can  retrieve 
guidance  about  the  reports  and  the  reports  themselves.  The  three  types  of  reports  are 
summary,  comparison,  and  stakeholder. 
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When  the  leader  requests  guidance  for  the  reports  the  system  provides  the  leader 
with  infonnation  about  each  of  the  reports.  When  a  leader  requests  a  report  the  system 
retrieves  the  raw  data  from  the  leader’s  account  fde,  compiles  it  per  the  requested  type  of 
report,  and  presents  it  to  the  leader  on  the  computer  screen.  Printable  versions  of  each 
report  are  also  available.  Once  the  leader  has  reports  in  hand  he  or  she  is  ready  to  take 
action  (see  Figure  18). 


Figure  18.  Level  1  DFD  for  process  6  (Take  action). 


As  necessary,  the  leader  can  retrieve  guidance  about  the  action  planning  process. 
In  this  case,  the  leader  requests  guidance  and  the  system  provides  the  leader  with  action 
planning  information.  Once  the  leader  understands  the  action  planning  process  he  or  she 
may  want  to  obtain  ideas  for  goals  setting.  To  do  this,  when  the  leader  requests 
suggested  actions  and  readings  the  system  provides  that  information  to  the  leader.  In  this 
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case,  the  system  provides  a  series  of  workbook  pages  that  provide  suggested  actions  and 
readings  to  improve  leadership  behaviors  in  a  specific  area.  For  instance,  a  leader  is 
deemed  weak  in  the  commitments  that  define  the  “Challenge  the  process”  leadership 
practice.  A  page  is  available  that  provides  suggested  actions  and  readings  to  improve  the 
commitments  and  behaviors  that  define  “Challenge  the  process”. 

Furthermore,  if  the  leader  requests  a  sample  or  template  action  plan,  the  system 
provides  copies  of  these  that  the  leader  can  edit  or  tailor  to  his  or  her  liking.  The  sample 
action  plan  provides  an  example  of  a  completed  action  plan.  The  template  action  plan 
provides  a  document  that  allows  the  leader  to  input  his  or  her  improvement  goals. 

The  system  was  designed  to  move  the  leader  through  the  developmental  feedback 
process  while  providing  the  necessary  tools  for  him  or  her  to  develop  an  action  plan.  At 
this  point  the  leader  has  reached  the  far  boundary  of  the  system  and  must  use  his  or  her 
intuition  to  put  the  action  plan  on  paper.  The  DFDs  showed  the  different  levels  of  the 
system,  their  functions,  and  the  flow  of  data  between  their  processes  and  data  stores. 

Database  models.  While  the  hierarchy  chart  and  DFDs  provided  the  overall 
system  structure,  the  database  models  were  used  to  represent  the  underlying  data 
structure.  This  section  presents  the  database  models  that  provide  a  multi-dimensional 
view  of  the  data.  It  also  presents  the  table  used  to  store  the  leaders’  account  information. 

Three-dimensional  (3-D)  database  models  were  developed  to  represent  the  cells  of 
data  in  an  array.  Two  models  were  created  to  represent  the  data  structure,  one  for  the 
LCI  feedback  data  and  one  for  the  stakeholder  feedback  data.  The  representation  of  the 
LCI  feedback  data  can  be  found  in  Figure  19. 
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Item 

Figure  19.  Three-dimensional  data  model  for  LCI-Self  and  Observer  data. 

A  number  (ranging  from  10  to  9999)  represents  each  leader  and  is  assigned  when  an 
account  is  created.  This  number  is  used  by  the  system  when  observers  complete  an 
assessment  and  ensures  the  feedback  data  is  sent  to  the  proper  leader’s  account  file.  The 
role  represents  the  relationship  of  the  observer  to  the  leader  (i.e.,  l=boss,  2=peer, 

3=direct  report,  4=indirect  report,  and  5=team  member).  The  self  and  boss  roles  will 
only  hold  one  row  of  data  while  the  other  roles  can  hold  numerous  rows  of  data 
(annotated  by  the  dotted  line).  This  signifies  that  the  leader  will  have  feedback  from 
more  than  one  person  in  each  of  those  categories.  The  assessment  item  columns  store  the 
individual  scores  for  each  item  of  the  assessments.  In  the  case  of  the  LCI,  there  are  48 
items.  If  there  are  three  or  more  observers  for  a  specific  category  the  system  should 
compile  and  present  the  information  in  a  report  when  requested.  Otherwise,  the  leader 
must  obtain  more  observers  to  view  the  compiled  data  in  a  report. 
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To  avoid  confusion,  a  separate  model  was  created  for  the  stakeholder  assessment. 
The  stakeholder  database  model  (found  in  Figure  20)  is  similar  to  the  LCI  database 
model. 


n 
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Figure  20.  Three-dimensional  data  model  for  stakeholder  data. 

The  feedback  data  is  still  represented  in  the  same  manner  but  the  difference  is  that  the 
stakeholder  assessment  only  has  24  items.  Furthermore,  there  is  only  one  role 
(stakeholder).  However,  at  least  three  stakeholders  must  input  information  for  the 
database  to  compile  the  data  and  present  it  in  a  stakeholder  report  to  the  leader. 

A  table  was  created  to  represent  each  leader’s  account  infonnation.  The  template 
table  is  found  in  Figure  2 1 .  The  account  ID  is  the  number  (10  to  9999)  assigned  to  the 
leader  when  he  or  she  created  the  account.  The  DateSubmitted  field  is  the  date  the 
account  was  created.  The  TimeSubmitted  is  the  time  the  account  was  created.  The 


FirstName  and  LastName  fields  are  the  leader’s  first  and  last  names.  The  Email  field  is 
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Accounts 

AccountID 

DateSubmitted 

TimeSubmitted 

FirstName 

LastName 

Email 

UserlD 

PWD 

Access 

Figure  21.  Accounts  table. 


the  leader’s  e-mail  address.  The  UserlD  is  the  user  identification  created  by  the  leader. 
The  PWD  field  is  the  leader’s  password.  The  Access  field  allows  access  to  the  account 
via  the  user  identification  and  password. 


Phase  3.  Physical  design 

During  the  physical  design  phase,  the  architectural  design  was  translated  into  an 
actual  system  design.  The  purpose  of  investigative  question  3  was  to  develop  an  actual 
system  design. 

Investigative  question  3.  How  does  the  system  architecture  affect  the  actual 
system  design? 

The  physical  design  of  the  system  provided  the  theoretical  framework  of  a  worthy  web- 
based  developmental  feedback  system  that  provided  the  necessary  tools  for  a  leader  to 
move  through  the  feedback  process  and  improve  his  or  her  leadership  behaviors. 

While  the  architectural  design  provided  functional  requirements  and  processes, 


the  system  design  provided  actual  system  specifications.  In  other  words,  specifications  of 
the  website,  feedback  instrument,  feedback  reports,  and  on-line  action  planning  tools 
were  developed  during  this  phase.  These  specifications  were  derived  from  the 
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requirements  baseline  and  system  architecture  and  resulted  in  a  series  of  templates. 

These  templates  provided  basic  specifications  for  content  of  each  web  page,  e-mail, 
report,  and  workbook  page  for  the  entire  system.  Only  the  requirements  for  each  page 
were  provided  on  the  template.  The  system  programmer  determined  graphics  design  and 
web  usability  features.  The  description  of  the  website  in  this  chapter  is  hierarchical  in 
nature  and  presents  the  information  as  the  leader  would  view  it  while  using  the  real 
system  for  the  first  time. 

Home  Page 

Since  the  system  was  web-based,  it  required  a  home  page.  The  home  page 
provides  evidence  that  the  user  has  reached  the  correct  website  and  is  available 
throughout  the  website.  It  also  provides  access  to  certain  functions  without  requiring 
secure  entry  to  the  system,  particularly  the  first  three  processes  defined  in  the 
architectural  design.  Therefore,  it  needed  links  for  the  user  to  obtain  background 
information,  create  an  account,  and  enter  the  system.  Beyond  that  it  also  needed  links  for 
the  user  to  exit  the  website  and  contact  the  researchers  for  help.  The  home  page  template 
can  be  found  in  Figure  22. 

The  title  in  Figure  22  was  developed  to  ensure  users  could  identify  they  had 
reached  the  correct  website.  To  further  enhance  the  users  experience  with  the  website,  it 
needed  a  standardized  background.  The  background  needed  to  display  the  title  of  the 
web  page  throughout  the  website.  The  title  displays  “The  Leadership  Commitments 
Inventory”  followed  by  “A  developmental  feedback  tool”  on  each  page  of  the  website. 
Furthermore,  the  researcher  felt  it  was  necessary  to  display  the  leadership  practices 
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Title 

Welcome  to  the  Leadership  Commitments  Inventory! 

A  developmental  feedback  tool 

Required  links 

About  the  LCI 
Commitment  Definitions 
Create  account 
Login 
Exit 

Contact  Us 

Figure  22.  Home  page  template  with  title  and  required  links. 

in  some  manner  throughout  the  site,  giving  it  an  aesthetic  appearance.  The  practices  are 
displayed  as  building  blocks  to  show  that  developmental  feedback  is  a  building  process  to 
better  leadership  behaviors.  The  building  blocks  template  can  be  found  in  Figure  23. 

The  building  blocks  were  the  only  graphic  designed  by  the  researcher.  The  system 
programmer  designed  all  other  graphics  and  color  schemes.  As  Figure  22  shows,  the 
home  page  was  designed  to  house  links  for  the  user  to  obtain  background  information, 
create  and  account,  and  enter  the  system.  It  also  provided  links  to  allow  the  user  to  exit 
the  system  and  contact  the  researchers  for  help. 

Get  background  information.  The  first  two  links  were  designed  to  provide 
background  information  for  the  user.  The  “About  the  LCI”  link  provided  background 
information  about  developmental  feedback  and  the  current  study.  It  also  provided 
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Figure  23.  Building  blocks  for  standardized  website  background. 

information  on  the  importance  of  participating  in  a  developmental  feedback  program  (see 
Appendix  B  for  “About  the  LCI”  page  template).  The  “Commitment  Definitions”  link 
was  designed  to  provide  information  about  the  relevant  leadership  practices  and 
commitments  that  are  the  focus  of  this  developmental  feedback  program.  The 
information  was  also  designed  to  show  users  the  leadership  behaviors  that  are  relevant  to 
their  organizations  (see  Appendix  C  for  the  “Commitment  Definitions”  page  template). 

Create  account.  The  “Create  account”  link  was  designed  to  allow  users  (leaders) 
to  create  a  secure  account  to  access  the  developmental  feedback  functions.  When  the 
user  selects  the  “Create  account”  link,  a  web  page  appears  that  requires  him  or  her  to 
enter  specific  information.  This  information  allows  the  system  to  identify  the  user  and 
allows  the  user  secure  access  to  the  system.  The  required  information  was  determined  to 
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be:  first  name,  last  name,  e-mail  address,  user  identification,  and  password.  The  user  is 
required  to  enter  his  or  her  password  twice  to  ensure  its  correctness.  Furthermore,  the 
system  only  allows  unique  user  names. 

Once  the  infonnation  is  submitted  the  user  receives  a  confirmation  that  the 
account  has  been  created.  This  message  reads,  “Your  account  has  been  created  and  an 
email  has  been  sent  to  the  email  address  specified  with  your  newly  created  account 
information.”  The  confirmation  provides  a  link  directly  to  the  login  area  so  the  user  can 
login  to  the  system.  At  this  time  the  system  assigns  the  user  with  a  user  identifier.  This 
is  a  number  between  10  and  9999  that  identifies  the  leader  to  the  database.  The  user  is 
unaware  of  this  number  until  he  or  she  attempts  to  obtain  feedback. 

Enter  system.  The  “Login”  link  provides  secure  access  to  the  system  and  its 
developmental  feedback  functions  (i.e.,  feedback,  reports,  and  action  planning).  When 
the  user  selects  this  link  they  are  directed  to  the  login  area.  Once  in  the  login  area  the 
user  is  required  to  enter  a  user  identification  and  password  and  submit  the  information.  If 
the  information  is  correct  he  or  she  will  be  directed  to  the  main  menu  page,  which  will  be 
described  later.  If  the  submitted  infonnation  is  incorrect  the  user  is  prompted  that  he  or 
she  has  entered  the  incorrect  information.  This  message  reads,  “Password  or  user 
identification  does  not  exist.”  At  this  point,  a  link  is  available  to  allow  the  user  to  return 
to  the  login  area. 

If  the  user  forgets  his  or  her  user  infonnation  or  password  he  or  she  is  able  to 
obtain  that  information  and  gain  access  to  the  system.  A  “Forgot  Password?”  link 
(located  in  the  login  area)  provides  this  function.  When  the  user  selects  this  link  he  or 
she  is  presented  a  web  page  with  the  statement  “Please  enter  the  following  information  in 
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order  to  log  into  the  Leadership  Commitments  Inventory  system.”  After  this  statement  is 
a  space  to  allow  the  user  to  enter  his  or  her  e-mail  address.  Once  the  user  enters  the  e- 
mail  address  and  clicks  the  “Retrieve  password”  link  he  or  she  is  directed  to  a  page  with 
the  statement  “Your  account  infonnation  and  password  has  now  been  sent  to  the  email 
address  specified.”  The  confirmation  also  provides  a  link  directly  to  the  login  area  so  the 
user  can  login  to  the  system.  At  this  point  the  user  is  able  to  retrieve  his  or  her  first  name, 
last  name,  and  user  identification  from  the  e-mail  message. 

Users  can  also  change  their  account  information.  The  “Change  Account 
Information”  link  (also  located  in  the  login  area)  allows  users  to  change  account 
information  without  creating  a  new  account,  which  allows  them  to  maintain  their  current 
feedback.  When  a  user  selects  this  link  he  or  she  is  directed  to  an  account  information 
page  that  displays  current  account  infonnation.  The  page  contains  a  prompt  that  tells  the 
user  to  change  desired  information.  Once  the  infonnation  is  submitted  the  user  receives  a 
confirmation  that  the  account  information  has  been  changed.  This  message  reads,  “Your 
account  has  been  updated  and  an  email  with  your  new  account  information  has  been  sent 
to  the  email  address  specified.”  The  confinnation  also  provides  a  link  directly  to  the 
login  area  so  the  user  can  login  to  the  system. 

Exit  and  contact  links.  If  at  any  time  the  user  wishes  to  exit  the  system,  he  or  she 
should  have  access  to  an  exit  link.  This  link  allows  the  user  to  exit  the  system  at  any 
time.  When  the  user  selects  this  link,  his  or  her  web  session  is  tenninated  immediately. 

Some  users  may  require  help  with  system  use  or  the  developmental  feedback 
process.  The  “Contact  us”  link  (in  Figure  23)  allows  the  user  to  contact  the  researcher 
and  system  administrators.  When  the  user  selects  this  link,  he  or  she  is  directed  to  an  e- 
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mail  template.  The  template  is  blank  except  for  the  “To”  block.  The  e-mail  address  in 
the  “To”  block  is  lci@afit.edu.  Once  the  user  completes  the  e-mail  template  and  clicks 
the  send  button,  the  message  is  sent  to  the  relevant  parties. 

The  home  page  was  designed  to  provide  information  and  functions  that  do  not 
require  secure  access  to  the  system.  To  obtain  secure  access  to  more  system  features  the 
user  is  required  to  login.  Once  the  user  logs  in  to  the  system,  he  or  she  has  access  to  the 
main  menu  options. 

Main  Menu  Page 

The  main  menu  page  must  be  accessed  through  the  home  page’s  login  function 
and  was  designed  to  allow  the  user  access  to  the  developmental  feedback  tools, 
particularly  processes  four  through  six  in  the  architectural  design.  To  get  started,  the  user 
can  obtain  guidance  about  the  developmental  feedback  process  and  how  to  use  the 
system.  Following  that,  he  or  she  can  obtain  feedback,  feedback  reports,  and  action 
planning  tools.  Therefore,  the  main  menu  has  links  to  guidance,  feedback,  reports,  and 
action  planning  tools.  To  ensure  the  user  could  still  exit  the  system  or  contact  the 
researchers  and  system  administrators  when  desired,  these  functions  were  also  required 
for  the  main  menu  page.  The  main  menu  page  template  can  be  found  in  Figure  24.  The 
title  and  web  page  background  for  the  main  menu  page  are  exactly  the  same  as  the  home 
page.  The  main  menu  page  allows  the  user  access  to  six  links.  These  links  are  presented 
on  the  page  in  the  same  order  as  Figure  24.  This  is  the  optimal  order  to  complete  the 
developmental  feedback  process  and  encourages  the  leader  to  select  links  in  the  proper 
order.  The  following  is  a  description  of  each  of  the  links. 
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Title 

Welcome  to  the  Leadership  Commitments  Inventory! 

A  developmental  feedback  tool 

Required  links 

How  to  get  started 
Get  feedback 
LCI  Reports 
Taking  action 
Exit 

Contact  us 

Figure  24.  Main  menu  page  template  with  title  and  required  links. 

Flow  to  get  started.  Users  can  access  guidance  about  the  developmental  feedback 
process  and  its  functions.  The  “How  to  get  started”  page  was  designed  to  provide  that 
information.  When  the  user  selects  the  appropriate  link,  he  or  she  is  directed  to  a  web 
page  that  contains  infonnation  about  the  developmental  feedback  process  and  its 
functions  (i.e.,  feedback,  reports,  action  planning).  This  page  provides  enough 
information  about  the  system  function  to  allow  the  user  to  feel  comfortable  with  using  the 
system  throughout  the  process.  The  “How  to  get  started”  page  information  can  be  found 
in  Appendix  D. 

Get  feedback.  Once  the  user  obtains  guidance  he  or  she  is  encouraged  to  obtain 
feedback.  The  “Get  feedback”  pages  were  designed  to  provide  all  the  necessary 
information  and  features  to  allow  the  user  to  obtain  accurate  developmental  feedback. 
When  the  user  selects  the  link  he  or  she  is  directed  to  a  page  that  contains  two  links: 
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“Create  a  new  assessment”  and  “Add  to  an  existing  assessment”.  The  template  for  this 
page  can  be  found  in  Appendix  E. 

If  this  is  the  first  time  the  user  has  visited  the  site  or  it  has  been  at  least  30  days 
since  the  user’s  last  assessment,  he  or  she  creates  a  new  assessment  by  selecting  the 
“Create  new  assessment”  link.  This  link  directs  them  to  the  “Get  Feedback”  instructions 
and  feedback  instrument  links.  If  this  is  not  the  user’s  first  assessment,  this  link  should 
prompt  the  database  to  transfer  all  existing  feedback  to  the  old  feedback  database.  The 
data  transferred  is  time  stamped  to  facilitate  future  feedback  retrieval.  The  user’s  account 
is  now  free  of  old  data  and  he  or  she  is  able  to  begin  a  new  assessment. 

If  the  user  would  like  to  add  more  assessments  to  the  existing  data  file  and  it  is 
within  30  days  of  the  original  request  for  feedback,  he  or  she  adds  to  the  existing 
assessment.  The  “Add  to  existing  assessment”  link  directs  him  or  her  to  the  “Get 
Feedback”  instructions  and  feedback  instrument  links.  The  difference  from  the  “Create 
new  assessment”  link  is  that  this  link  opens  the  existing  assessment  database  and  inputs 
data  accordingly.  No  old  data  is  removed  from  the  existing  database. 

As  stated,  once  the  user  selects  the  “Create  new  assessment”  or  “Add  to  existing 
assessment”  link,  he  or  she  is  directed  to  the  “Get  Feedback”  instructions  and  options 
page  (Appendix  F).  The  instructions  are  available  on  this  page  along  with  three  links  to 
obtain  feedback.  These  links  allow  the  user  to  obtain  feedback  via  the  FCI-Self,  FCI- 
Observer,  and  stakeholder  assessment. 

To  obtain  a  self-assessment  the  leader  selects  the  FCI-Self  link.  When  the  user 
selects  this  link  he  or  she  is  directed  to  the  FCI-Self  feedback  instrument.  The  template 
for  the  FCI-Self  can  be  found  in  Appendix  G.  The  data  from  this  instrument  is  fed  into 
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the  user’s  account  file.  The  user  is  provided  with  a  cover  page  that  explains  the  purpose, 
use,  and  participation  requirements  of  the  LCI-Self.  Following  the  cover  page  is  the 
instructions  page,  which  explains  that  the  LCI-Self  was  designed  to  allow  leaders  to 
document  perceptions  of  their  own  behavior  and  that  the  inputs  to  the  electronic  survey 
go  directly  to  the  survey  control  point.  Once  the  user  has  read  the  instructions  he  or  she 
is  able  to  click  a  continue  button  that  goes  to  the  first  page  of  the  survey.  The  user  is 
prompted  to  “Please  read  each  statement  carefully  and  indicate  the  extent  to  which  you 
engage  in  each  of  the  behaviors.  Answer  in  terms  of  how  you  typically  act  with  the 
people  in  your  unit  and  on  behalf  of  your  unit.” 

Each  page  of  the  survey  instrument  contains  no  more  than  four  items.  Each  item 
has  a  full  response  scale  with  an  option  button  for  each  response  choice.  Each  page  has  a 
continue  link  that  brings  up  the  next  page  of  the  survey.  Each  page  also  tells  the  user 
how  far  along  in  the  survey  he  or  she  is.  For  example,  a  message  is  visible  at  the  bottom 
of  a  page  that  states,  “50%  completed”.  The  final  page  of  the  survey  allows  the  user  to 
input  comments  or  feedback.  Once  the  user  has  completed  the  LCI-Self  he  or  she  should 
receive  a  confirmation  that  the  information  has  been  submitted.  Likewise,  he  or  she  is 
able  to  select  a  link  that  takes  them  back  to  the  “Get  feedback”  instructions  and  options 
page. 

Once  the  user  returns  to  the  “Get  feedback”  instruction  and  options  page,  he  or 
she  is  able  to  request  feedback  from  observers  (i.e.,  their  boss,  peers,  direct  reports, 
indirect  reports,  and  team  members).  To  request  feedback  from  observers  the  user  must 
select  the  LCI-Observer  link.  When  the  user  selects  this  link  he  or  she  is  directed  to  an  e¬ 


mail  template  (Appendix  H).  At  this  point  the  system  automatically  inserts  the  user’s 
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identifier  number  and  signature  block  into  the  e-mail  template.  The  user  must  then  input 
his  or  her  observers’  e-mail  addresses  in  the  “To”  block  of  the  template.  After  that,  in  the 
body  of  the  e-mail  template  and  where  prompted,  the  user  inputs  the  names  of  his  or  her 
observers  and  their  relationships.  Once  the  user  has  accomplished  these  items  he  or  she 
sends  the  e-mail. 

When  the  observer  receives  the  e-mail  he  or  she  must  select  the  link  contained  in 
the  body  of  the  e-mail  to  access  the  LCI-Observer  (Appendix  I).  The  LCI-Observer  is 
similar  to  the  LCI-Self  except  that  when  the  cover  page  appears  on  the  screen  the 
observer  must  input  the  leader’s  user  identifier  number  (from  the  e-mail)  and  the 
relationship  to  the  leader.  The  possible  relationships  (also  called  roles)  to  the  leader  are 
boss,  peer,  direct  report,  indirect  report  and  team  member.  Each  choice  has  an  option 
button  that  allows  the  observer  to  select  the  appropriate  relationship.  Once  the  correct 
information  is  submitted  the  observer  is  directed  to  a  page  that  presents  the  information 
he  or  she  entered  and  asks  whether  or  not  it  is  the  proper  leader  and  relationship.  The 
observer  selects  “yes”  or  “no”  and  proceeds  to  the  survey  instrument.  If  the  observer 
enters  “no”  they  are  directed  to  the  cover  page  to  re-enter  the  user  identifier  number  and 
relationship.  If  the  observer  fails  to  enter  the  user  identifier  number  or  relationship  the 
system  prompts  the  observer  to  enter  the  infonnation.  Once  the  LCI-Observer  is 
completed  the  data  is  input  to  the  appropriate  user’s  (leader’s)  account  file. 

If  the  user  wishes  to  obtain  stakeholder  data  he  or  she  selects  the  stakeholder  link 
on  the  “Get  feedback”  page.  When  the  user  selects  the  link  he  or  she  is  directed  to  an  e- 
mail  template  similar  to  the  LCI-Observer  e-mail  template.  This  template  is  available  in 
Appendix  J.  There  are  a  couple  differences  in  the  process.  Lirst,  when  the  stakeholder 
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selects  the  link  from  the  e-mail  he  or  she  is  directed  to  the  stakeholder  assessment  (see 
Appendix  K).  Second,  when  the  stakeholder  assessment  cover  page  appears  on  the 
screen,  the  stakeholder  is  only  required  to  enter  the  leader’s  user  identifier  number  (not  a 
relationship).  Once  the  survey  is  completed  and  submitted  the  data  is  input  to  the 
appropriate  user’s  (leader’s)  account  file. 

When  the  user  has  completed  a  self-assessment  and  has  received  feedback  from 
three  or  more  observers  in  a  particular  category,  he  or  she  is  able  to  obtain  summary  and 
comparison  feedback  reports.  If  the  user  has  received  feedback  from  three  or  more 
stakeholders  he  or  she  is  able  to  obtain  a  stakeholder  report. 

LCI  reports.  The  “LCI  Reports”  link  was  designed  to  provide  the  necessary 
guidance  and  reports  and  allow  users  to  view  and  analyze  their  feedback.  When  a  user 
selects  the  link  he  or  she  is  directed  to  the  “LCI  Reports”  page  that  contains  assessment 
response  infonnation  and  four  links  (see  Figure  25). 

The  title  contains  assessment  response  information  that  shows  the  user  how  many  total 
responses  he  or  she  currently  has  and  how  many  responses  he  or  she  has  for  each 
feedback  category.  This  allows  the  user  to  detennine  whether  or  not  he  or  she  has  data  in 
a  specific  category  and  whether  or  not  he  or  she  can  request  a  report.  The  letters  A-F  are 
counts  for  each  assessment  category.  The  “SUM(A:E)”  adds  up  the  total  responses  for 
LCI-Observer  survey. 

To  obtain  reports  it  is  first  recommended  that  the  user  obtain  guidance  about  the 
reports  and  their  interpretation.  The  “Understanding  the  Reports”  page  provides  this 
guidance.  When  the  user  selects  the  “Understanding  the  Reports”  link,  he  or  she  is 
directed  to  a  page  that  contains  information  about  each  of  the  three  reports  and  how  to 
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Title 

LCI  Reports 

Your  current  leadership  assessment  is  based  on  SUM(A:E)  responses. 

A  Boss;  B  Peer;  C  Direct  Report;  D  Indirect  Report;  E  Team  Member 
You  also  have  F  stakeholder  responses. 

Required  links 
Understanding  the  Reports 
Summary  Report 
Comparison  Reports 
Stakeholder  Report 

Figure  25.  LCI  Reports  page  template  with  title  and  required  links. 

obtain  them.  This  page  also  contains  infonnation  about  reading  and  interpreting  the 
reports.  The  “Understanding  the  Reports”  page  information  can  be  found  in  Appendix  L. 

A  summary  of  the  feedback  scores  is  obtained  from  the  “Summary  Report”  link. 
The  summary  report  includes  summaries  of  the  6  practices,  12  commitments,  and  48 
items  (behaviors).  The  report  also  contains  a  column  for  each  feedback  category  or  role 
(i.e.,  self,  boss,  peer,  team  member,  direct  and  indirect  report)  and  is  labeled  according  to 
the  template  in  Appendix  M. 

When  the  user  first  selects  the  “Summary  report”  link  he  or  she  are  directed  to  the 
“Summary  Feedback”  page  that  displays  his  or  her  summary  data  for  the  practices  and 
commitments.  To  protect  the  observers’  anonymity,  all  scores  listed  under  a  specific 
observer  category  (with  the  exception  of  the  boss  category)  are  the  average  scores  of  at 
least  three  observers.  If  there  is  a  category  that  is  not  applicable  to  the  user  or  a  category 
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that  has  not  received  feedback  from  three  observers,  no  information  appears  in  that 
category’s  column.  The  user  is  able  to  click  on  the  empty  column’s  header  to  hide  the 
column.  Furthermore,  if  a  user  wishes  to  obtain  the  practice  or  commitment  definitions 
for  reference,  the  summary  report  contains  a  link  to  the  “Commitment  Definitions”  page 
(Appendix  C). 

The  user  is  also  able  to  get  summary  information  for  each  specific  item 
(leadership  behavior).  To  accomplish  this,  each  practice  or  commitment  header  is  a  link 
to  a  summary  of  behaviors  for  that  practice  and  its  commitments  (see  Appendix  N).  The 
“Behavior  Feedback”  page  contains  scores  for  each  of  the  individual  behaviors.  As 
stated  above,  scores  listed  under  observer  categories  (with  exception  to  the  boss  category) 
are  the  average  scores  of  the  observers’  responses  for  each  item.  On  this  page,  each 
practice  or  commitment  header  is  a  hyperlink  that  takes  the  user  to  the  definitions  page 
for  reference,  if  necessary. 

The  comparison  reports  allow  the  user  to  relate  perceptions  of  his  or  her 
leadership  behaviors  to  the  perceptions  of  the  observers.  The  reports  contrast  the  user’s 
score  with  the  score  of  the  user’s  boss,  or  with  the  average  score  of  the  user’s  peers,  team 
members,  direct  reports,  and  indirect  reports  for  each  of  the  12  leadership  commitments. 
In  other  words,  five  reports  are  available  (i.e.,  self  to  boss,  self  to  peers,  self  to  direct 
reports,  etc.).  Each  comparison  is  a  different  report. 

The  “Comparison  Reports”  link  directs  the  user  to  the  “Comparison  Reports” 
menu.  This  menu  has  links  to  the  five  available  reports.  Each  link  directs  the  user  to  the 
chosen  report.  The  template  for  the  “Comparison  Reports”  menu  page  is  found  in  Figure 
26  and  the  comparison  report  template  can  be  found  in  Appendix  O. 
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Title 

Comparison  Reports 
Required  links 
Self  to  Boss 
Self  to  Peers 
Self  to  Direct  Reports 
Self  to  Indirect  Reports 
Self  to  Team  members 

Figure  26.  Comparison  reports  page  template  with  title  and  required  links. 

The  reports  provide  the  user  with  the  self-assessment  score  and  the  average 
observer  score  for  each  of  the  12  commitments.  It  also  provides  a  pictorial  bar  graph  of 
the  scores.  Each  score  is  computed  by  taking  the  average  across  all  items  relating  to  that 
commitment.  The  observers’  scores  are  combined  to  ensure  anonymity.  The  report  also 
provides  the  difference  between  the  self-assessment  and  observer  scores.  To  compute  the 
difference  the  self-assessment  score  is  subtracted  from  the  observer  score  for  that 
commitment.  Negative  numbers  are  acceptable.  The  commitments  are  ranked  on  the 
report  based  on  the  difference  from  lowest  to  highest.  The  final  column  of  the  report 
provides  an  indication  of  the  variability  of  the  responses  and  is  computed  as  the  standard 
deviation  of  the  observer  scores. 

The  final  report  available  to  the  user  is  the  stakeholder  report.  This  report 
presents  the  scores  and  comments  from  the  stakeholder  assessment  and  is  located  at  the 
“Stakeholder  Report”  link  on  the  “LCI  Reports”  page  (previously  depicted  in  Figure  25). 
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When  the  user  selects  the  link  he  or  she  is  directed  to  the  report  (see  Appendix  P  for 
report  template). 

This  report  provides  an  item-by-item  summary  of  the  stakeholder  assessment. 

The  scores  are  computed  by  calculating  the  average  score  of  the  observers’  responses  for 
each  item.  The  report  also  provides  an  indication  of  the  variability  of  the  responses, 
which  is  computed  as  the  standard  deviation  of  the  observers’  scores.  The  last  section  of 
the  report  provides  a  summary  of  the  written  comments  as  provided  by  the  observers. 

This  section  described  the  reports  that  are  available  to  the  user.  It  should  be  noted 
that  the  system  should  be  able  to  produce  a  printable  version  of  each  report.  The 
printable  versions  of  the  report  should  be  presented  in  an  organized  and  easily  readable 
manner.  Once  the  user  obtains  the  desired  reports  he  or  she  is  able  to  proceed  to  the  next 
step  in  the  process,  action  planning. 

Taking  Action.  The  “Taking  Action”  link  was  designed  to  provide  some 
necessary  tools  to  encourage  the  user  to  set  goals  and  take  action  from  the  feedback. 
Particularly,  the  tools  include  guidance  for  taking  action,  suggested  actions  and  readings 
for  goal  setting,  and  action  plans.  When  the  user  selects  the  link  he  or  she  is  directed  to 
the  “Taking  Action”  page.  The  template  for  this  page  can  be  found  in  Figure  27. 

The  user  should  have  access  to  action  planning  infonnation  and  the  “Guidance  for 
Taking  Action”  link  provides  this  function.  When  the  user  selects  this  link  he  or  she  is 
directed  to  the  “Guidance  on  Taking  Action”  page.  This  page  provides  information  about 
building  an  action  plan,  sharing  the  feedback  and  action  plan  with  others,  and  following 
through  with  the  action  plan.  The  template  and  information  for  the  guidance  page  can  be 
found  at  Appendix  Q. 
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Title 

Taking  Action 
Required  links 

Guidance  for  Taking  Action 
Suggested  Actions  and  Readings 
Build  an  Action  Plan 

Figure  27.  Taking  Action  page  template  with  title  and  required  links. 

To  access  the  suggested  actions  and  readings  for  each  of  the  six  leadership 
practices,  the  user  selects  the  “Suggested  Actions  and  Readings”  page  link.  This  link 
directs  the  user  to  the  “Suggested  Actions  and  Readings”  menu  page.  The  template  for 
this  page  can  be  found  in  Figure  28. 


Title 

Suggested  Actions  and  Readings 
Required  links 
Model  the  Way 
Inspire  a  Shared  Vision 
Challenge  the  Process 
Enable  Others  to  Act 
Encourage  the  Heart 
Enjoy  the  Workplace 

Figure  28.  Suggested  Actions  and  Readings  page  template  with  title  and  required  links. 
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Each  of  the  required  links  directs  the  user  to  a  separate  window  and  a  page  that 
provides  the  definition  of  the  practice,  some  suggested  actions  to  improve  behaviors 
relevant  to  that  practice,  and  some  suggested  readings  that  focus  on  the  leadership 
behaviors  pertaining  to  that  practice.  Furthennore,  each  link  name  contains  the  practice 
and  its  commitments.  Each  page  is  downloadable  and  printable.  Finally,  there  should  be 
a  printable  version  of  the  entire  workbook.  A  “Suggested  actions  and  readings”  page  for 
each  practice  can  be  found  in  Appendix  R. 

The  final  link  on  the  “Taking  Action”  page  provides  information  for  the  user  to 
build  an  action  plan.  Specifically,  the  “Build  an  Action  Plan”  link  directs  the  user  to  the 
“Build  an  Action  Plan”  page,  which  contains  two  links.  The  template  for  the  “Build  an 
Action  Plan”  page  can  be  found  in  Figure  29. 

Title 

Build  an  Action  Plan 
Required  links 
Sample  Action  Plan 
Action  Plan  Template 

Figure  29.  Build  an  Action  Plan  page  template  with  title  and  required  links. 

When  the  user  selects  the  “Sample  Action  Plan”  link,  he  or  she  is  directed  to  a  separate 
window  and  page  containing  the  sample  action  plan.  When  the  user  selects  the  “Action 
Plan  Template”  link  he  or  she  is  directed  to  a  separate  window  and  page  containing  a 
blank  action  plan  template.  These  actions  plan  are  also  downloadable  and  printable. 
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Sample  and  template  action  plans  can  be  found  in  Appendixes  S  and  T,  respectively. 

The  “Taking  Action”  page  was  designed  to  provide  the  user  with  the  proper  tools 
to  set  goals  using  an  action  plan.  The  “Taking  Action”  function  also  stands  as  the  far 
boundary  of  the  system.  From  here,  it  is  up  to  the  user  to  follow  the  action  plan  and  take 
action  to  change  leadership  behaviors. 

Phase  IV.  Implementation 

The  implementation  phase  consisted  of  system  building,  testing,  installation, 
operations,  and  post-implementation  review.  Investigative  question  4  was  used  to 
construct  the  actual  system. 

Investigative  question  4.  How  do  you  translate  the  architectural  and  physical 
designs  into  a  working  information  system? 

This  question  was  answered  with  the  first  four  steps  of  the  implementation  phase  (i.e., 
system  building,  testing,  installation,  and  operations). 

System  Building 

The  graduate  school’s  programmers  programmed  all  system  requirements.  The 
system  was  developed  and  implemented  using  Dream  Weaver  (Version  MX)  and  Cold 
Fusion  (Version  5.0)  software  packages,  created  by  Macromedia.  The  database  was 
created  using  SQL  Server  2000  by  Microsoft.  The  school’s  programmers  designed  all 
graphics  using  Fireworks  by  Macromedia.  The  system  can  be  accessed  on  the  Internet  at 
http  ://en.  afit.  edu/  env/lci/. 

Currently  the  feedback  collecting  and  action  planning  functions  are  working  and 
available  on  the  website.  As  designed,  the  leader  accesses  the  main  system  functions 


through  the  home  page  and  main  menu  page.  Screen  shots  of  these  pages  are  found  in 
Figure  30. 
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Home  page 


3  LCI  Survey  -  Microsoft  Internet  Explorer  provided  by  AFIT 

J*  Microphone  [pi Tools  ^Handwriting 

File  Edit  View  Favorites  Tools  Help 

4*  Back  ^  (Q  [^ji  -jffi  |  ^Search  HfeJ  Favorites  Media  ^  0 

5*  ifl  6D  -  3 

Address  |  http:J/en, aPit.edu/env/lci/ 


Leadership  Commitments  Inventory 
A  developmental  feedback  tool 
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gjmrnitmer  : 
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Osol&sQQte 


Main  menu  page 


Figure  30.  Screen  shots  of  web  site  home  page  and  main  menu  page. 
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A  leader  can  get  background  information,  create  an  account,  log  in  to  the  system, 
exit  the  system,  and  contact  the  system  administrator  and  researcher.  Both  feedback 
instruments  collect  data  and  input  it  to  the  database.  The  system  also  provides  action¬ 
planning  tools  for  the  leader  to  use. 

There  are  some  features  of  the  system  that  have  not  been  implemented.  The 
feedback  collecting  function  that  allows  the  leader  to  create  a  new  assessment  or  add  to 
an  existing  assessment  has  not  been  implemented.  Currently,  the  system  only  allows 
leaders  to  create  a  new  assessment  when  they  create  a  new  account.  From  then  on  the 
system  adds  to  a  leader’s  current  assessment.  The  stakeholder  assessment  function  is  not 
available.  Finally,  the  system  will  not  compile  feedback  data  and  present  any  of  the 
feedback  reports.  Reports,  when  necessary,  are  developed  manually  by  the  researcher 
and  sent  to  leaders  via  e-mail.  With  most  of  the  system  functioning,  the  researcher 
proceeded  with  testing  to  obtain  evidence  of  the  system’s  functionality  and  ease  of  use. 
Testing 

The  results  of  the  testing  phase  revealed  only  minor  problems  or  concerns  with 
functionality.  The  problems  were  identified  as  follows.  The  get  feedback  e-mail 
template  needed  to  be  modified  so  that  observers  could  get  to  the  survey  link  quickly, 
without  reading  a  lengthy  e-mail.  The  exit  link  was  not  functioning.  The  background 
“building  blocks”  were  too  dark.  Some  of  the  survey  questions  were  difficult  to  put  into 
the  correct  perspective.  Finally,  there  was  no  exit  button  at  the  end  of  the  observer 
assessment.  These  problems  were  all  corrected  before  putting  the  system  into  full 
operation.  Particularly,  the  e-mail  template  was  modified,  the  exit  link  was  activated,  the 
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“building  blocks”  were  lightened,  the  relevant  survey  questions  were  re -worded,  and  an 
exit  link  was  added  to  the  end  of  the  observer  assessment. 

The  system  was  perfonning  ah  other  functions  as  designed.  Ease  of  use  data  was 
available  from  35  observers  and  the  results  were  as  follows.  The  skewness  and  kurtosis 
values  of  the  distribution  (S=- 3.5;  K=  \  6.4)  fell  outside  normality  range,  indicating  that 
statistics  other  than  the  mean  and  standard  deviation  should  be  reported.  The  mode  for 
the  ease  of  use  scale  was  7.00  and  the  median  was  6.50.  The  minimum  and  maximum 
values  were  2.25  and  7.00,  respectively.  The  minimum  value  only  occurred  one  time. 
The  34  other  values  were  5.50  or  greater,  or  toward  the  agreement  end  of  the  7-point 
scale.  There  were  no  comments  given  by  users  having  difficulty  with  system  operation. 
These  results  indicated  that  the  web  survey  was  easy  to  use,  particularly  from  the 
observers’  point  of  view.  Since  the  testing  phase  revealed  that  the  system  was 
functioning  as  designed  and  was  seemingly  easy  to  use,  it  was  possible  to  install  ah 
available  functions. 

Installation 

The  school’s  programmers  installed  the  system  functions  on  the  web  site  as  the 
researcher  approved  them.  Ah  functions  described  during  the  system-building  step  were 
installed  before  the  testing  phase.  After  the  testing  phase,  the  school’s  programmers 
installed  ah  updated  functions.  To  facilitate  the  next  step  of  this  phase,  system 
operations,  the  test  feedback  data  was  removed  from  the  database.  Leaders’  account  files 
remained  in  tact  and  leaders  who  had  participated  in  the  test  phase  did  not  have  to  create 
a  new  account  to  participate  in  the  operational  phase.  Once  this  step  was  complete  the 
system  was  ready  for  operations. 
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Operations 

The  system  was  fielded  to  students  and  faculty  at  two  different  graduate  schools 
in  the  area.  Leaders  were  encouraged  to  use  the  system  to  obtain  feedback  concerning 
their  leadership  behaviors.  To  date,  sixty-eight  leaders  have  used  the  system  to  complete 
a  self-assessment  and  obtain  feedback.  Two  hundred  ten  observers  have  fully  completed 
the  LCI-  Observer.  Data  were  compiled  from  these  people  to  assess  the  reliability  and 
validity  of  the  feedback  instruments.  The  system  continues  to  operate  without  the  full 
reports  function.  At  this  time,  the  reports  will  be  compiled  manually  and  presented  to 
leaders  via  e-mail.  Once  programmed,  this  function  will  be  available  automatically. 
Post-implementation  Review 

Once  the  system  was  operational  it  was  necessary  to  obtain  information  on  the 
program  and  infonnation  system.  Particularly,  the  researcher  wanted  to  obtain  data  on 
the  perceived  utility  of  the  feedback  and  usability  of  the  system.  Investigative  question  5 
was  used  to  assess  these  variables. 

Investigative  question  5.  How  do  you  assess  the  perceived  utility  of  the  feedback 
and  usability  of  the  infonnation  system? 

To  measure  feedback  utility,  the  researcher  relied  on  comments  provided  by 
leaders  and  observers  at  the  end  of  the  assessments.  All  comments  were  analyzed  for 
perceptions  of  feedback  utility  from  the  leader  and  observer  perspectives.  Furthermore, 
the  LCI-Observer  contained  two  items  at  the  end  of  the  assessment  that  were  used  to 
measure  perceived  utility  from  the  observer  perspective.  An  analysis  of  the  descriptive 
statistics  from  these  items  combined  with  the  analysis  of  the  comments  was  used  to 
formulate  conclusions  about  the  utility  (i.e.,  usefulness)  of  the  feedback. 
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Of  the  comments  submitted  by  leaders  and  observers,  fifteen  dealt  with  feedback 
utility.  One  observer  felt  the  survey  would  “identify  personal  strengths  and  leadership 
qualities.”  This  was  the  only  positive  comment  about  utility.  Of  the  remaining  fourteen 
comments,  ten  comments  addressed  that  observers  felt  the  survey  did  not  tailor  to  the 
relationship  they  had  with  the  ratee  and  needed  a  “not  applicable”  or  “no  opinion”  scale 
option. 

The  remaining  four  comments  were  individual  recommendations  and  were 
difficult  to  categorize.  Those  comments  are  described  individually  as  follows.  One 
observer  addressed  that  the  ratee  did  not  have  the  discretion  to  perform  certain  leadership 
behaviors.  Another  observer  felt  the  feedback  instrument  missed  “subtleties  and 
nuances”  related  to  leadership.  One  person  suggested  that  an  appropriate  item  at  the  end 
of  the  instrument  might  be,  “I  found  this  survey  applicable  to  the  ratee.”  That  person  also 
suggested  providing  some  alternatives  that  allow  the  observer  to  show  a  behavior  was  not 
observed.  That  person  felt  a  “not  observed”  scale  option  was  different  from  “sometimes” 
or  “almost  never  observed.”  Finally,  an  observer  felt  the  survey  should  have  a  “when 
appropriate”  scale  option  because  some  situations  or  positions  prevent  a  leader  from 
performing  specific  behaviors. 

Perceived  utility  was  measured  from  the  observer  viewpoint  using  two  items  at 
the  end  of  the  LCI-Observer  assessment.  The  perceived  utility  items  were  “I  feel  I  have 
worked  long  enough  with  this  person  to  adequately  assess  his/her  leadership  behaviors” 
and  “I  believe  the  information  collected  by  this  survey  can  help  leaders  become  more 
effective.”  Skewness  values  for  the  items  were  -1.3  and  -1.2,  respectively.  Generally, 
the  data  were  skewed  to  the  agreement  end  of  the  7-point  Likert  scale,  as  expected.  This 
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range  of  skewness  allowed  the  researcher  to  assume  that  the  data  were  normally 
distributed  (Neter,  J.,  Kutner,  M.,  Nachtscheim,  C.,  &  Wasserman,  W.,  1996).  Kurtosis 
values  for  the  items  were  1 .47  and  1 .69,  respectively.  From  this,  the  researcher 
concluded  that  the  data  have  slightly  higher  peaks  than  a  normal  distribution.  Still,  all  of 
the  data  fell  within  the  acceptable  range  for  the  nonnality  assumption. 

Means  for  the  two  items  were  5.58  and  5.59,  respectively.  Standard  deviations 
were  1.43  and  1.29.  Results  indicated  that  observers  tended  to  agree  that  the  feedback 
they  provided  would  be  useful  to  the  leaders.  Particularly,  observers  felt  they  had  worked 
with  the  leader  long  enough  to  accurately  assess  his  or  her  leadership  behaviors  and  felt 
the  feedback  provided  could  help  leaders  become  more  effective. 

To  measure  system  usability,  the  researcher  measured  the  observers’  perceptions 
of  the  system’s  ease  of  use.  Any  comments  provided  by  leaders  and  observers  that 
related  to  ease  of  use  issues  were  noted.  The  LCI-Observer  contained  four  items  at  the 
end  of  the  assessment  that  were  used  to  measure  perceived  usability  from  the  observer 
perspective.  An  analysis  of  the  comments  and  descriptive  statistics  was  used  to 
determine  perceived  usability. 

Eight  comments  indicated  there  were  no  major  problems  with  the  system’s  ease  of 
use.  Two  comments  indicated  the  survey  was  easy  to  understand  and  complete.  Three 
comments  indicated  that  the  text  in  the  survey  was  still  hard  to  read.  Two  of  those 
comments  indicated  the  “building  blocks”  in  the  background  were  too  dark.  Two  people 
were  unable  to  or  did  not  know  how  to  return  to  the  previous  page  once  they  clicked  the 
“continue”  button  at  the  bottom  of  the  survey  page.  One  of  these  people  suggested 
putting  a  back  button  next  to  the  continue  button.  A  final  comment  indicated  that  the 
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beginning  of  the  survey  was  too  slow.  The  observer  suggested  no  solution  to  this 
problem. 

Perceived  usability  (i.e.,  ease  of  use)  was  also  measured  from  the  observer 
viewpoint  using  the  ease  of  use  scale  at  the  end  of  the  LCI-Observer  assessment.  Ease  of 
use  data  was  available  from  210  observers  and  the  results  were  as  follows.  The  skewness 
value  for  the  scale  was  -2.29,  indicating  the  data  were  skewed  to  the  agreement  end  of  the 
7-point  Likert  scale,  as  expected.  This  range  of  skewness  allowed  the  researcher  to 
assume  that  the  data  were  normally  distributed  (Neter,  J.,  Kutner,  M.,  Nachtscheim,  C.,  & 
Wasserman,  W.,  1996).  The  kurtosis  value  for  the  scale  was  8.62.  From  this,  the 
researcher  concluded  that  the  data  have  a  high  peak.  However,  the  data  fell  outside  the 
acceptable  range  for  the  normality  assumption,  indicating  that  statistics  other  than  the 
mean  and  standard  deviation  should  be  reported. 

The  mode  for  the  ease  of  use  scale  was  6.00  and  the  median  was  6.25.  The 
minimum  and  maximum  values  were  2.00  and  7.00,  respectively.  Two  hundred  of  the 
209  scale  values  were  5.00  or  greater.  These  results  provided  evidence  that  observers 
tended  to  agree  that  the  system  was  easy  to  use.  The  combination  of  the  comments  and 
descriptive  statistics  lead  the  researcher  to  conclude  that,  from  the  observer’s  perspective, 
the  system  was  easy  to  use. 

This  concluded  the  implementation  phase  and  requirements  for  the  study.  The 
maintenance  phase  is  ongoing  and  was  beyond  the  scope  of  this  study. 

Chapter  Summary 

This  chapter  contained  results  of  the  planning,  analysis,  physical  design,  and 
implementation  phases  of  the  information  system  development  life  cycle.  The  planning 
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phase  conducted  the  initial  investigation  for  the  study  and  detennined  that  the  study  was 
feasible.  Once  it  was  detennined  that  the  study  was  feasible,  the  researcher  moved  to  the 
analysis  phase  of  the  life  cycle.  During  this  phase,  the  problem  was  redefined  and  the 
requirements  baseline  and  system  architectural  design  were  developed.  The  physical 
design  was  derived  from  the  requirements  baseline  and  system  architecture  and  resulted 
in  a  series  of  templates.  Particularly,  the  physical  design  was  made  up  of  a  series  of 
templates  and  specifications  for  the  website,  feedback  instruments,  feedback  reports,  and 
action  planning  tools.  The  physical  design  was  subsequently  used  to  build  the  actual 
system.  Many  steps  of  the  implementation  phase  were  accomplished.  While  the  system 
was  not  fully  implemented,  enough  of  the  system  was  implemented  for  leaders  to  obtain 
feedback.  Reports  can  be  manually  generated  until  the  automatic  reports  function  is 
implemented.  Comments  and  data  from  the  initial  feedback  utility  and  system  usability 
assessment  indicate  that  the  feedback  is  useful  and  the  system  is  easy  to  use,  particularly 
from  the  observers’  perspective.  The  implementation  of  the  system  completed  the 
requirements  for  this  study. 
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V.  Limitations,  Conclusions,  and  Recommendations 

This  chapter  provides  some  limitations  to  the  study.  It  further  provides  conclusions 
and  recommendations  to  ensure  successful  implementation  and  maintenance  of  the  web- 
based  developmental  feedback  program.  It  also  highlights  some  lessons  learned  from  the 
system  development  process. 


Limitations 

No  study  is  without  limitations.  This  study  is  no  exception  and  four  limitations 
were  discovered.  First,  the  data  structure  design  could  have  been  designed  with  more 
detail.  While  the  architectural  design  provided  a  good  view  of  the  underlying  data 
structure,  the  researcher  did  not  fully  develop  the  physical  design  of  the  data  structure.  A 
physical  design  would  have  showed  how  the  system  queried  the  database  and  how  the 
database  compiled  data  for  feedback  reports.  A  physical  design  structure  would  have 
given  the  programmers  an  easier  time  developing  the  database  and  implementing  the 
reports  function. 

Second,  a  good  cross  section  of  Air  Force  officers  and  a  diverse  group  of  MBA 
students  from  first  line  supervisors  to  middle  managers  used  the  system.  Feedback  from 
their  perspectives  was  important  to  the  study  because  they  are  a  large  target  group  for  the 
system.  However,  the  researcher  was  not  able  to  get  any  senior  managers  to  use  the 
system.  It  would  have  been  beneficial  to  have  additional  feedback  about  the  system  and 
its  function  from  their  perspectives. 

The  third  limitation  dealt  with  the  utility  and  usability  assessment.  The  utility  and 


ease  of  use  items  from  the  observer  assessment  were  not  included  in  the  self-assessment. 
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The  original  objective  was  to  complete  a  full  utility  and  usability  assessment  using  a 
separate  questionnaire.  Since  the  full  assessment  was  not  accomplished,  the  researcher 
missed  an  opportunity  to  obtain  some  utility  and  usability  data  from  the  leaders’ 
perspectives  by  including  the  items  in  the  self-assessment. 

Finally,  the  researcher  could  not  assess  whether  or  not  the  reports,  action  planning 
tools,  and  stakeholder  functions  were  valid  because  they  had  not  been  implemented. 
Without  these  functions  in  place  and  functioning,  the  researcher  had  no  way  of  assessing 
if  the  functions  actually  helped  move  leaders  through  the  feedback  process  to  improve 
their  leadership  behaviors.  With  the  limitations  noted,  the  researcher  was  now  able  to 
present  some  conclusions  and  recommendations. 

Conclusions  and  Recommendations 

Conclusion  1 

The  results  of  four  phases  of  the  information  systems  development  life  cycle  were 
presented  in  Chapter  IV.  The  overall  objective  of  this  study  was  to  develop  an  effective 
web-based  system  for  the  developmental  feedback  programs  of  Aeronautical  Systems 
Center,  Air  Force  Security  Assistance  Center,  and  Air  Force  Materiel  Command. 
Specifically,  this  study  developed  a  web-based  system  that  performed  three  distinctive 
functions:  1)  on-line  collection  instruments  for  self  and  observer  assessments,  2) 
automatic  feedback  data  entry  for  each  leader  and  across  leaders  for  the  researcher,  and 
3)  on-line  workbook  to  aid  leader  self-analysis  and  development  of  action  plans.  These 
functions  were  designed  to  move  leaders  through  the  feedback  process  to  improve  their 
leadership  behaviors. 
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The  fact  that  the  system  did  not  compile  feedback  data  and  create  reports  was,  as 
stated  before,  a  major  limitation  to  this  study.  The  reports  function  was  necessary  for  the 
researcher  to  fully  assess  the  utility  of  the  feedback  and  usability  of  the  system  from  the 
leaders’  perspectives.  Furthermore,  it  would  have  provided  evidence  about  the  overall 
effectiveness  of  the  program. 

Recommendation  1 

The  first  recommendation  is  that  the  reports  function  be  implemented  (as 
designed)  as  soon  as  possible,  along  with  the  rest  of  the  relevant  functions  that  were  not 
implemented.  Once  the  reports  function  is  implemented,  a  large,  diverse  group  of 
personnel  from  many  different  organizations  should  use  the  system  to  obtain  feedback 
and  reports.  When  leaders  have  viewed  the  results  of  their  feedback  and  made  attempts 
to  set  goals  and  develop  action  plans,  they  should  be  able  to  provide  information  on  their 
perceptions  of  the  feedback  process  and  system  function  during  the  full  utility  and 
usability  study. 

The  feedback  utility  study  could  be  used  to  determine  the  extent  to  which  leaders 
perceived  the  feedback  provided  was  accurate  and  useful.  The  utility  study  could  also 
determine  whether  or  not  the  program  provided  the  leader  with  the  necessary  tools  to  set 
goals  and  improve  their  leadership  behaviors.  The  system  usability  study  could  be  used 
to  determine  the  extent  to  which  leaders  were  satisfied  with  the  system  and  its  function. 
The  study  could  determine  the  extent  to  which  leaders  found  the  system  easy  to  use. 
Furthermore,  the  study  could  determine  whether  or  not  leaders  were  satisfied  with  the 
quality  of  the  infonnation  provided  by  the  system  and  interface  with  the  system. 
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Results  from  the  limited  utility  and  usability  assessment  items  indicated  that,  in 
general,  observers  felt  that  the  feedback  provided  by  the  system  could  be  useful  to  leaders 
trying  to  improve  their  effectiveness.  There  was  also  evidence  that  observers  felt  the  web 
survey  used  to  obtain  the  feedback  was  easy  to  use.  At  this  point  some  of  the  feedback 
utility  and  usability  comments  provided  by  observers  should  be  addressed. 

Conclusion  2 

Many  observers  felt  that  a  “not  applicable”,  “not  observed”,  or  “no  opinion”  scale 
option  should  be  implemented.  Even  if  observers  had  not  observed  the  leader  performing 
certain  behaviors,  it  seems  they  were  reluctant  to  choose  the  “almost  never  observed” 
scale  option  because  they  felt  it  would  reflect  badly  on  the  person  being  assessed.  They 
felt  a  “not  applicable”,  “not  observed”,  or  “no  opinion”  choice  would  result  in  more 
accurate  feedback.  This  choice  would  portray  to  the  leader  that  they  did  not  have  the 
opportunity  to  observe  a  specific  behavior  or  they  felt  the  leader  did  not  have  the 
discretion  to  perform  a  specific  behavior. 

One  observer  also  suggested  adding  a  “when  appropriate”  scale  option.  This 
person  cited  specific  examples  of  behaviors  that  leaders  could  only  perfonn  when 
appropriate  (e.g.,  “I  encourage  the  open  exchange  of  ideas  and  information”).  Once 
again,  this  observer  was  reluctant  to  provide  truthful  feedback  because  he  or  she  felt  it 
would  reflect  badly  upon  the  leader.  A  “when  appropriate”  choice  would  make  the 
observer  feel  like  they  provided  more  accurate  feedback  than  if  they  used  one  of  the  other 
options. 
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Recommendation  2 

This  researcher  recommends  a  “not  observed”  option  replace  the  “almost  never  “ 
option  to  allow  the  observers  to  annotate  they  had  not  observed  a  specific  behavior.  This 
might  serve  to  ease  their  minds  that  they  are  not  damaging  the  leader’s  reputation. 

Along  the  same  lines,  one  observer  suggested  that  a  “when  appropriate”  option 
should  be  available.  This  researcher  agreed  with  the  observer  that  some  leadership 
behaviors  should  only  be  used  when  appropriate.  This  is  particularly  true  in  the  military 
environment.  For  instance,  some  leaders  need  to  be  directive  and  decisive  in  certain 
situations  that  do  not  allow  them  to  gather  the  opinions  of  their  coworkers  and 
subordinates  before  they  make  the  decision.  This  does  not  necessarily  mean  they  do  not 
enable  their  people,  it  simply  means  decisive  action  had  to  be  taken  at  that  particular 
time.  Therefore,  I  recommend  that  the  instructions  for  the  LCI  should  be  changed  to 
instruct  leaders  and  observers  to  rate  the  extent  to  which  the  behaviors  are  done  when 
appropriate.  The  self-assessment  instructions  should  read,  “Please  read  each  statement 
and  indicate  the  extent  to  which  you  engage  in  each  of  the  behaviors  when  appropriate. 
Answer  in  terms  of  how  you  typically  act  with  and  on  behalf  of  your  unit.”  The  observer 
assessment  instructions  should  read,  “Please  read  each  statement  and  indicate  the  extent 
to  which  you  have  observed  this  person  doing  each  of  the  behaviors  when  appropriate. 
Answer  in  terms  of  how  you  typically  acts  with  you,  with  people  in  your  unit,  and  on 
behalf  of  you  and  your  unit.” 

Along  with  the  aforementioned  updates  to  the  instrument  rating  scale,  this 
researcher  felt  that  leaders  and  observers  should  be  in  the  right  frame  of  mind  before 
participating  in  the  program.  Thus,  a  short  training  video  should  be  added  to  the  main 
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menu  page.  This  video  would  appear  the  first  time  a  leader  logs  in  to  the  system  and 
would  always  be  available  via  the  main  menu.  The  video  would  feature  a  senior  leader 
that  would  welcome  leaders  to  the  program.  The  leader  would  highlight  the  importance 
and  benefits  of  the  program  and  its  impact  on  the  organization  while  encouraging  leaders 
to  seek  honest  feedback.  The  leader  would  also  encourage  leaders  to  make  personal 
contact  with  their  observers  before  seeking  feedback  to  get  the  observers  in  the  right 
frame  of  mind  for  the  program.  Leaders  should  encourage  observers  during  a  meeting  or 
phone  conversation  to  provide  honest  feedback.  They  should  also  explain  that  low 
ratings  are  not  necessarily  a  bad  reflection  of  a  person’s  leadership  skills  and  will  not 
damage  a  reputation,  given  that  the  program  is  developmental  and  the  feedback  is 
confidential. 

To  further  get  leaders  in  the  right  frame  of  mind  for  the  program  a  small  section 
should  be  added  to  the  “Understanding  the  reports”  page.  This  section  would  prompt  a 
leader  to  ask  certain  questions  when  interpreting  reports.  Specifically,  the  leader  could 
ask  “Is  this  behavior  applicable  to  me?”  or  “Does  this  behavior  matter  (given  the  context 
of  my  duty)?”  If  an  area  is  not  applicable,  the  leader  could  discard  the  results.  A  better 
solution,  however,  would  be  for  the  leader  to  share  the  results  of  that  feedback  with 
observers  to  ensure  the  area  is  not  applicable.  This  action  would  provide  evidence  that 
observers  also  felt  the  behavior  was  not  applicable  and  did  not  observe  the  leader 
performing  the  behavior,  prompting  them  to  rate  him  or  her  low  in  those  areas. 

These  updates  to  the  leadership  commitments  inventory  and  leader/observer 
interaction,  if  implemented,  could  go  a  long  way  to  encourage  observers  to  provide 
accurate  and  useful  feedback  by  using  the  entire  rating  scale. 
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Conclusion  3 

The  fact  that  the  ease  of  use  data  and  comments  were  generally  positive  was  a 
direct  result  of  designing  the  system  with  usability  in  mind.  Changes  and  updates  were 
made  throughout  the  design  process  to  accommodate  usability  and  “customer” 
requirements.  The  researcher  worked  closely  with  the  system  programmers  throughout 
all  phases  of  the  development  life  cycle  to  ensure  the  system  was  self-sufficient  and  easy 
for  leaders  to  use. 

Recommendation  3 

Therefore,  this  researcher  recommends  that  during  the  maintenance  phase  the 
system  be  continually  monitored  and  updated  to  improve  system  functions  and  usability, 
moving  it  closer  to  its  intended  purpose.  During  this  phase,  leaders  and  observers  could 
continue  to  provide  information  about  the  feedback  process  and  system.  The  program 
could  be  updated  on  a  periodic  basis,  implementing  recommended  changes  from  the 
feedback  received.  The  result,  a  usable  system  that  remains  self-sufficient  and  provides 
leaders  with  the  proper  development  tools,  could  go  a  long  way  to  ensure  success  for 
many  leaders  and  their  organizations. 

Final  recommendations 

To  start  the  maintenance  phase  the  researcher  performed  a  final  assessment  of  the 
system  and  its  functions.  One  item  surfaced  that  could  potentially  improve  the  usability 
of  the  system  and  its  function.  Another  item  identified  a  potential  application  for  the 
system. 

First,  the  “get  feedback”  function  should  be  split  into  two  separate  functions.  One 
function  would  provide  instructions  and  a  link  for  the  leader  to  complete  a  self- 
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assessment.  The  other  function  would  provide  instructions  and  links  for  the  leader  to 
obtain  feedback  from  observers  and  stakeholders.  This  breaks  up  the  lengthy  “get 
feedback  instructions  and  options”  page  and  provides  a  more  logical  structure  for  leaders 
to  obtain  feedback.  These  choices  would  be  implemented  to  the  main  menu  page  and  all 
other  functions  would  work  as  designed. 

Second,  the  web-based  developmental  feedback  program  could  have  an 
application  here  at  the  graduate  school.  Instructors  could  use  this  program  to  obtain 
feedback  from  their  bosses,  students,  and  fellow  faculty  members  and  use  it  improve  their 
leadership  behaviors.  To  do  this,  the  current  LCI  feedback  instrument  would  need  to  be 
updated  to  reflect  behaviors  relevant  to  instructors  and  their  profession.  This  application 
would  allow  quick  and  easy  compilation  of  data  and  would  provide  students  with  an 
anonymous  voice.  Instructors  could  use  the  feedback  to  improve  items  such  as  course 
layout  and  class  lectures.  Students  would  respect  the  fact  that  instructors  are  actively 
trying  to  improve  areas  of  weakness. 


Lessons  Learned 

There  were  many  lessons  learned  throughout  the  systems  development  process. 
First  and  foremost,  the  life  cycle  should  be  treated  as  a  dynamic  and  flexible  process.  At 
times,  it  seemed  like  program  and  system  requirements  were  changing  daily.  It  was 
necessary  to  work  closely  with  the  system  programmers  to  ensure  they  were  kept  aware 
of  the  changes  and  required  updates.  However,  due  to  the  dynamic  nature  of  the  process, 
changes  will  be  necessary.  To  avoid  numerous  changes,  present  items  for 
implementation  only  after  they  have  been  thoroughly  reviewed.  Too  many  changes  will 
lengthen  the  development  time  line. 
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Closing  Remarks 

The  purpose  of  this  study  was  to  develop  a  web-based  developmental  feedback 
program  that  would  provide  the  necessary  tools  to  allow  a  leader  to  improve  his  or  her 
leadership  behaviors.  The  success  of  a  program  such  as  this  begins  with  its  ability  to 
provide  feedback  that  leaders  accept  as  accurate.  Otherwise,  they  never  get  past  the  first 
step  of  the  feedback  process.  However,  no  program  can  be  effective  without  buy  in  from 
the  people  who  are  using  it.  Leaders  must  want  to  improve  their  leadership  behaviors  and 
organizations  must  encourage  their  leaders  to  improve. 

The  system  was  designed  to  provide  an  effective  and  inexpensive  platfonn  for 
organizations  to  conduct  developmental  feedback  programs.  It  could  also  provide  a 
template  for  organizations  that  desire  to  design  and  build  their  own  programs.  This  study 
sets  merely  the  groundwork  for  a  multitude  of  follow-on  study  in  this  area. 
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Appendix  A:  Requirements  Baseline 

1.  System  Requirements.  The  system  must  be  self-sufficient  and  allow  the  leader  to 
securely  move  through  the  feedback  process  with  little  or  no  outside  assistance.  It  should 
also  provide  information  that  will  help  the  leader  determine  whether  or  not  he  or  she 
wants  to  participate  in  the  program. 

1 . 1  Background  infonnation.  The  system  should  provide  background  information  on  the 
current  study  and  developmental  feedback. 

1.1.1  Leadership  Commitments  Inventory.  The  system  should  provide  background 
information  on  the  Leadership  Commitments  Inventory  and  how  it  relates  to  a 
developmental  feedback  program.  The  system  should  also  provide  infonnation  on  the 
background  of  this  study. 

1.1.2  Definitions.  The  system  should  provide  definitions  of  the  relevant  leadership 
behaviors  for  this  developmental  feedback  program. 

1 .2  System  overview.  The  system  should  provide  information  that  will  familiarize  a 
leader  with  system  features  and  use. 

1 .3  Confidentiality.  The  system  must  maintain  confidentiality  of  information. 
Furthermore,  the  user  must  be  confident  that  their  information  is  safe.  The  system  must 
allow  leaders  to  have  secure  access  to  the  system  and  their  feedback  infonnation. 

1.3.1  Access  to  account  infonnation.  Leaders  should  have  unrestricted  access  to  their 
secure  accounts  and  information. 

1 .4  Contact.  The  system  should  provide  a  contact  e-mail  account  that  will  allow  users  to 
contact  the  research  team  or  technical  support  with  problems,  questions,  or  concerns. 

1.5  Exit  system.  The  system  should  allow  the  leader  to  exit  the  system  when  desired. 

2.  Feedback  Collecting.  The  system  should  collect  accurate  feedback. 

2.1  Feedback  instruments.  The  instrument  deployed  with  the  system  should  be  reliable, 
valid,  and  focus  on  relevant  leadership  behaviors.  It  should  also  be  designed  for  self  and 
observer  assessments. 

2.1.1  Leadership  Commitments  Inventory  (Self).  The  feedback  instrument  (based  on 
two  reliable  and  valid  instruments)  was  designed  for  leaders  to  perfonn  a  self-assessment 
of  their  leadership  behaviors.  Its  reliability  and  validity  will  be  tested  with  system  use. 

2.1.2  Leadership  Commitments  Inventory  (Observer).  This  feedback  instrument  (based 
on  two  reliable  and  valid  instruments)  was  designed  for  observers  (i.e.,  boss,  peers,  direct 
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subordinate,  indirect  subordinate,  and  team  members)  to  rate  a  leader’s  leadership 
behaviors.  Its  reliability  and  validity  will  be  tested  with  system  use. 

2.1.3  Stakeholder  assessment.  This  feedback  instrument  was  designed  for  stakeholders 
(customers)  to  rate  the  leader’s  organization’s  performance  and  customer  service.  The 
stakeholder  assessment  was  also  designed  to  test  criterion  validity  of  the  LCI. 

2.2  Anonymity.  System  must  ensure  raters’ anonymity.  To  do  this,  each  leader  must 
have  at  least  three  observers  for  a  specific  category  (excluding  boss  category)  rate  his  or 
her  leadership  behaviors  using  the  LCI-Observer. 

2.3  Confidentiality.  System  must  maintain  confidentiality  of  all  feedback  infonnation. 
The  graduate  school  must  maintain  authority  over  all  system  information. 

2.4  Guidance.  System  must  provide  guidance  for  completing  a  self-assessment  and 
obtaining  feedback  from  observers  and  stakeholders. 

3.  Results  Reporting.  The  system  must  provide  clear,  useful,  and  understandable  reports 
with  relevant  feedback  information. 

3.1  Summary  Report.  The  summary  report  should  provide  a  graphical  representation  of 
summarized  results  at  the  practice,  commitment,  and  behavior  level  for  the  self  category 
and  each  observer  category.  The  summary  practice  score  is  the  average  of  its  two 
commitment  scores.  The  commitment  score  is  the  average  of  its  four  items  (behaviors). 
The  behavior  score  is  represented  as  an  average  of  the  observer  scores  for  that  behavior. 
There  should  be  summary  practice,  commitment,  and  behavior  scores  for  each  category 
(i.e.,  boss,  peers,  subordinates,  team  members). 

3.2  Comparison  Report.  The  comparison  report  should  provide  a  graphical 
representation  allowing  the  leader  to  compare  his  or  her  perceptions  with  observers’ 
perceptions.  In  other  words,  the  leader  should  be  able  to  compare  his  or  her  scores  with 
average  observer  scores  at  the  commitment  level  for  each  category  (e.g.,  self  to  boss,  self 
to  peers,  etc.). 

3.3  Stakeholder  Report.  The  stakeholder  report  should  provide  a  graphical 
representation  of  summarized  results  of  the  stakeholder  assessment.  The  summary  scores 
are  the  average  scores  of  each  assessment  item  across  the  stakeholders. 

3.4  Anonymity.  System  must  ensure  raters’  anonymity.  To  do  this,  the  system  must  not 
provide  summary  or  comparison  infonnation  for  an  observer  category  until  at  least  three 
observers  in  that  category  have  provided  feedback  (excluding  boss  category).  The 
system  must  not  produce  a  stakeholder  report  until  at  least  three  stakeholders  have 
provided  feedback. 


3.5  Guidance.  The  system  must  provide  guidance  about  the  three  types  of  reports,  what 
they  are  used  for,  and  how  to  retrieve  and  read  them. 
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4.  Action  planning.  The  system  must  provide  action-planning  tools  that  will  allow 
leaders  to  set  goals  and  improve  their  leadership  behaviors. 

4.1  Workbook  or  Self  Study  Guide.  The  workbook  should  assist  leaders  in  the  action 
planning  process. 

4.1.1  Guidance  on  the  action  planning  process.  This  function  should  provide  guidance 
for  leaders  to  build,  share,  and  implement  an  action  plan. 

4.1.2  Action  plan  template.  This  function  should  provide  the  leader  with  a  template 
action  plan  so  they  do  not  have  to  create  one  of  their  own.  The  action  plan  should  give 
the  leader  the  capability  to  set  improvement  goals. 

4.1.3  Suggested  actions  and  readings.  This  function  should  provide  lists  of  suggested 
actions  and  readings  that  leaders  can  use  to  improve  specific  leadership  areas.  This 
function  should  be  easy  to  navigate  and  the  infonnation  should  be  provided  in  an 
organized  manner  (e.g.,  at  the  practice  level). 
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Appendix  B:  About  the  LCI page 


Background  on  the  LCI 

Aeronautical  Systems  Center  (ASC)  and  Air  Force  Security  Assistance  Center 
(AFSAC)  senior  leaders  placed  significant  emphasis  on  providing  tools  to  enhance 
supervisors’  performance  as  leaders.  They  concluded  that  feedback  was  an  integral  part 
of  supervisory  maturity  and  began  an  effort  to  institute  a  developmental  feedback 
program.  A  primary  goal  for  the  program  was  to  be  consistent  with  the  six  leadership 
principles  presented  by  ASC  during  their  annual  Leadership  Symposiums.  Five  of  the  six 
ASC  leadership  principles  are  based  on  the  practices  described  in  Kouzes  and  Posner’s 
(1995),  The  Leadership  Challenge.  The  sixth  leadership  principle  was  created  to  capture 
the  important  leadership  behaviors  that  encourage  having  fun  in  the  workplace. 

The  ASC  Developmental  Feedback  team  reviewed  several  commercial  products 
that  could  be  used  to  garner  information  from  subordinates  including  the  Leadership 
Practices  Inventory  (LPI)  developed  by  Kouzes  and  Posner  (1995).  The  cost  of 
commercial  surveys,  inflexibility  of  formats,  and  insufficient  coverage  of  all  six 
leadership  principles  led  ASC  to  explore  the  possibility  of  Air  Force  Institute  of 
Technology  (AFIT)  assistance.  AFIT  responded  by  developing  observer  and  self-rating 
versions  of  a  feedback  instrument  based  on  the  commitments  presented  in  Kouzes  and 
Posner’s  (1995),  The  Leadership  Challenge.  The  resulting  instrument  was  named  the 
Leadership  Commitments  Inventory  (LCI).  The  AFIT  research  team  also  designed  a 
pilot  program  to  assess  the  reliability  and  validity  of  the  feedback  and  self-assessment 
instruments.  Over  100  supervisors  and  nearly  1,000  direct  reports  participated  in  the 
pilot  program.  A  status  brief  of  the  pilot  program  is  available  at 
http  ://en.  afit.  edu/  env/lcistatusbrief/. 

This  year  the  program  has  been  extended  into  a  developmental  feedback  program 
and  is  now  available  to  any  organization  that  wants  to  use  it.  Leaders  can  get  feedback 
from  direct  and  indirect  reports,  team  members,  peers,  and  bosses  using  an  updated 
version  of  the  LCI  developed  from  the  results  of  the  pilot  program.  The  program  remains 
purely  developmental  and  specific  results  are  kept  confidential.  No  one  in  the  leadership 
chain  will  ask  for  or  see  any  leader’s  feedback. 

Your  participation  matters 

Assessment  by  supervisors,  peers,  team  members,  and  subordinates  can  be 
extremely  informative.  Each  person  you  work  with  has  a  unique,  and  often  essential, 
perspective  on  your  effectiveness.  People  observe  your  interactions  with  others  and  most 
importantly  with  themselves.  The  Leadership  Commitments  Inventory  (LCI)  is  based  on 
behaviors  that  all  supervisors  can  do  and  can  learn  to  do  better.  You  can  learn  more 
about  these  behaviors  by  reading  Kouzes  and  Posner’s  book  The  Leadership  Challenge, 
or  any  of  the  many  other  references  we  have  provided  at  this  web  site.  As  you  request 
feedback  from  the  members  of  your  unit,  please  encourage  them  to  give  you  an  honest 
assessment.  Let  them  know  that  their  anonymity  is  assured,  and  that  no  one  in  your 
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management  chain  will  receive  copies  of  your  feedback.  Also  let  them  know  that  when 
you  get  your  personalized  feedback  you  will  spend  some  time  making  sense  of  it  and  then 
share  your  feedback  and  action  plans  with  them.  You  will  be  given  tools  to  help  guide 
you  through  the  process.  Taking  a  few  minutes  up  front  to  share  your  views  and 
encouragement  on  developmental  feedback  will  give  your  employees  and  coworkers  the 
confidence  that  their  opinions  matter  to  you. 

Most  of  us  do  not  understand  the  impact  our  actions  have  on  others  and  we  are  not 
always  sure  our  work  is  appreciated.  Developmental  feedback  is  one  way  to  get  some  of 
this  information.  This  program  will  not  address  all  of  the  issues  involved  with  being  a 
leader,  but  it  is  a  great  start.  The  challenges  are  enormous.  However,  we  believe  that  it 
will  bring  each  of  us  one  step  closer  to  what  the  Air  Force  expects  of  us  as  today’s  and 
tomorrow’s  leaders. 
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Appendix  C:  Leadership  Practice  and  Commitment  Definitions  page 

The  Leadership  Commitment  Inventory  (LCI)  is  designed  to  measure  the  six  leadership 
practices  at  the  commitment  level  and  determines  the  extent  to  which  the  leader  engages 
in  these  leadership  behaviors. 

Model  the  way 

Leaders  who  Model  the  Way  demonstrate  high  standards  and  establish  clear 
expectations  for  individual  performance.  To  model  the  way,  the  leader  must  Share 
Personal  Values  and  Set  the  Example.  Leaders  Share  Personal  Values  after  they  learn 
what  makes  them  tick.  They  clarify  and  communicate  their  values  so  that  others  know 
what  they  stand  for.  Sets  the  Example  refers  to  daily  behaviors  that  demonstrate  and 
teach  those  values  and  standards.  Leaders  who  do  what  they  say,  spend  time  on  what  is 
important,  and  lead  by  example,  appear  credible  to  others. 

Inspire  a  shared  vision 

Leaders  who  Inspire  a  Shared  Vision  convey  a  vivid  image  of  the  future  and 
develop  a  general  understanding  of  that  vision  among  the  organization’s  members. 
Commitments  for  this  practice  are  Create  a  Vision  and  Attract  Others  to  a  Common 
Purpose.  Create  a  Vision  refers  to  leadership  behaviors  that  convey  a  vivid  image  of  the 
organization’s  future.  Attract  Others  to  a  Common  Purpose  refers  to  leadership 
behaviors  that  show  and  communicate  how  aspirations  are  mutually  beneficial  to  work 
group  members  and  the  organization. 

Challenge  the  process 

Leaders  who  Challenge  the  Process  encourage  their  people  to  search  for 
opportunities  to  change  the  status  quo,  experiment,  take  risks,  and  leam  from  the 
mistakes.  The  commitments  that  define  this  practice  are  Seeks  Innovation  and  Take 
Risks  and  Learn  From  Mistakes.  Seek  Innovation  refers  to  leadership  behaviors  that 
search  for  and  encourage  others  to  search  for  opportunities  to  improve  the  efficiency  and 
effectiveness  of  the  organization.  Take  Risks  and  Learn  From  Mistakes  refers  to 
leadership  behaviors  that  create  opportunities  for  team  members  to  experiment  with  new 
ideas  in  order  to  gain  critical  knowledge  about  the  best  ways  to  add  value  to  the 
customer. 

Enable  others  to  act 


Leaders  who  Enable  Others  to  Act  foster  teamwork  among  their  organization’s 
members,  and  create  an  environment  of  mutual  respect  and  trust.  These  leaders  focus  on 
the  commitments  of  Promote  Cooperation  and  Empower.  Promote  Cooperation  refers 
to  leadership  behaviors  that  encourage  the  open  exchange  of  information  and  ideas 
among  work  group  members,  promote  good  working  relationships  with  outside 
organizations,  and  build  teams  both  within  and  outside  our  organization.  Empower 
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refers  to  leadership  behaviors  that  provide  members  with  the  necessary  resources, 
support,  and  skills  to  take  control  of  their  jobs  and  make  significant  inputs  to  the 
organization. 

Encourage  the  heart 

Leaders  who  Encourage  the  Heart  set  high  expectations,  recognize  individuals 
for  their  progress  and  contributions,  provide  rewards  for  exceptional  perfonnance,  and 
celebrate  the  accomplishments  of  the  work  group.  They  Recognize  Individual 
Contributions  and  reward  individual  progress  and  contributions  that  meet  high  standards 
of  performance.  They  also  Celebrate  Team  Accomplishments,  personally  highlighting 
and  recognizing  the  work  group’s  attainment  of  key  objectives  and  goals. 

Enjoy  the  workplace 

Leaders  who  Enjoy  the  Workplace  create  a  playfully  productive  atmosphere  at 
work  by  encouraging  humor  and  promoting  fun  activities.  These  leaders  Allow  Humor 
to  Reduce  Stress  and  Boredom  by  encouraging  humor  to  break  tension  and  create  an 
enjoyable  workplace.  Likewise,  they  Promote  Fun  Activities  to  Relax  and  Unwind 
and  encourage  creative  and  fun  activities  to  increase  morale  and  job  satisfaction. 
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Appendix  D:  How  to  Get  Started  page 

Thank  you  for  participating  in  the  Developmental  Feedback  Program.  This  web- 
based  version  has  been  designed  to  make  the  entire  feedback  process  easier  for  leaders 
looking  to  improve  their  leadership  skills.  This  page  offers  guidance  to  get  you  started. 
There  are  three  steps  to  the  developmental  feedback  process:  getting  feedback,  analyzing 
feedback  reports,  and  building  an  action  plan. 

Getting  feedback 

Your  first  step  is  to  get  feedback.  From  the  main  menu,  simply  click  the  “Get 
Feedback”  link  and  you  will  be  instructed  to  first  complete  a  self-assessment  using  the 
Leadership  Commitments  Inventory-Self  (a.k.a.  LCI-Self).  This  self-assessment  will 
give  you  a  basis  to  compare  your  own  perceptions  of  your  leadership  behaviors  with  the 
perceptions  of  others.  It  will  also  give  you  an  idea  of  the  leadership  behaviors  being 
measured.  After  completing  the  self-assessment,  you  should  return  to  the  “Get 
Feedback”  page  to  request  feedback  from  your  boss,  peers,  team  members,  and  direct  and 
indirect  reports.  An  indirect  report  is  defined  as  someone  who  is  two  levels  below  you  in 
the  organization. 

There  are  two  types  of  feedback  to  request.  First,  you  can  request  feedback  via 
the  LCI-Observer.  It  measures  the  same  behaviors  as  your  self-assessment,  but  from  an 
observer’s  perspective.  Once  two  people  in  each  of  the  peer,  team  member,  and 
direct/indirect  report  categories  have  completed  the  LCI-Observer  survey,  you  will  be 
able  to  get  reports  for  that  category.  For  example,  if  two  peers  complete  the  LCI- 
Observer  you  will  be  able  to  view  a  report  of  their  feedback.  The  same  holds  for  each 
observer  category.  The  other  type  of  feedback  you  can  request  is  stakeholder  feedback. 
The  stakeholder  survey  allows  people  to  assess  your  unit’s  performance.  These 
responses  are  also  anonymous  and  you  will  be  able  to  view  a  report  of  the  feedback  as 
soon  as  two  people  complete  the  stakeholder  survey.  There  are  no  categories  for  this 
survey. 

LCI  Reports 

Your  next  step  in  the  process  is  to  view  and  analyze  your  feedback  reports.  These 
reports  are  available  via  the  LCI  reports  icon  in  the  main  menu.  Once  you’ve  read 
“Understanding  the  Reports”  on  the  reports  page,  choose  the  reports  you  want  to  view 
and  analyze.  Print  out  the  reports  you  want  as  these  will  help  you  build  your  action  plan. 

Taking  Action 

Once  you  and  your  respondents  have  measured  (through  the  survey  instrument) 
the  extent  you  act  in  ways  consistent  with  the  12  leadership  commitments,  the  last  steps 
in  the  feedback  process  are  to  build  an  action  plan  based  on  the  feedback,  share  the 
feedback  and  action  plan  with  others,  and  follow  through  with  the  action  plan.  From  the 
main  menu,  click  the  “Taking  Action”  link  to  find  ways  you  can  implement  changes  to 
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your  leadership  behaviors.  There  is  guidance  on  building  an  action  plan  and  sharing 
feedback  with  others.  Also  there  are  lists  of  suggested  readings  that  highlight  leadership 
behaviors  consistent  with  the  12  commitments. 

We  hope  this  gives  you  a  good  idea  of  how  to  navigate  your  way  through  the  LCI 
web  site  and  the  leadership  development  process.  If  you  have  any  questions  or  problems 
during  the  process,  please  feel  free  to  contact  the  Air  Force  Institute  of  Technology 
Leadership  Commitments  Inventory  (LCI)  Research  Team  at  lci@alit.edu. .  We  are 
always  looking  for  ways  to  improve  the  developmental  feedback  process  and  this  web 
site.  If  you  have  any  feedback  for  us,  please  contact  us.  Thank  you. 
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Appendix  E:  Get  feedback  page 


Title 

Get  feedback 

(Prompt  statement)  If  this  is  the  first  time  you  have  visited  the  site  OR  it  has  been  30 
days  since  your  last  assessment,  please  click  the  link  below. 

(Required  link)  Create  new  assessment 

(Prompt  statement)  If  you  would  like  to  add  more  assessments  to  your  existing  data  file 
and  it  is  within  30  days  of  your  original  request  for  feedback,  please  click  the  link  below. 

(Required  link)  Add  to  existing  assessment 

If  you  have  questions  during  this  process  please  contact  the  AFIT  Research  Team  at 
lci@afit.edu. 
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Appendix  F:  Get  feedback  instructions  and  options  page 


Title 

Get  feedback 

We  recommend  you  first  complete  a  self-assessment.  This  will  allow  you  to  compare 
your  own  perceptions  of  your  leadership  behaviors  with  the  perceptions  of  others.  To 
complete  a  self-assessment,  please  choose  the  LCI-Self  icon  below. 

Next,  you  can  request  feedback  from  your  boss,  peers,  team  members,  and  direct  and 
indirect  reports  by  choosing  the  “LCI-Observer”  link  below.  Enter  the  addresses  of 
those  you  want  feedback  from  and  send  the  e-mail  containing  the  link  to  the  survey. 
Those  who  receive  the  e-mail  will  have  the  option  of  providing  you  feedback  on  your 
leadership  behaviors  through  the  LCI-Observer  survey.  The  responses  will  be 
automatically  tabulated  and  available  for  you  via  the  LCI  Reports  icon  in  the  main  menu. 
In  order  to  protect  the  anonymity  of  your  peers,  team  members,  and  direct/indirect 
reports,  two  people  must  complete  the  survey  in  a  particular  category  before  reports  for 
that  category  will  be  available.  So  if  you  want  feedback  for  a  specific  category  please 
send  the  e-mail  to  at  least  two  people  represented  by  that  category.  If  a  category  doesn’t 
apply,  that’s  OK,  you  can  still  get  feedback  and  reports  from  people  in  the  other 
categories. 

Finally,  you  can  request  stakeholder  feedback  from  people.  The  stakeholder  survey 
allows  people  who  receive  products  or  services  from  your  unit  to  provide  feedback  on 
your  unit’s  performance.  Request  this  feedback  by  choosing  the  “Stakeholder”  link 
below.  You  will  perform  the  same  steps  as  above.  Once  again,  the  responses  will  be 
automatically  tabulated  and  available  for  you  via  the  LCI  Reports  icon  in  the  main  menu. 
To  protect  anonymity  in  this  case,  two  people  must  complete  this  survey  before  reports 
will  be  available.  There  is  no  category  specific  feedback. 

If  you  have  questions  during  this  process  please  contact  the  AFIT  Research  Team  at 
lci@afit.edu. 

Required  Links 

LCI-Self 

LCI-Observer 

Stakeholder 


Appendix  G:  LCI  Self  Assessment 

Cover  Page 
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Air  Force  Institute  of  Technology 


Leadership  Commitments  Inventory 


Self 


Privacy  Notice 

The  following  information  is  provided  as  required  by  the  Privacy  Act  of  1974: 

Purpose:  To  obtain  information  regarding  leaders'  perceptions  of  their  own  leadership  behaviors. 

Routine  Use:  The  survey  results  will  be  used  to  provide  developmental  feedback  for  individual 
leaders  No  analysis  of  individual  responses  will  be  conducted  and  only  members  of  the  Air 
Force  Institute  of  Technology  research  team  will  be  permitted  access  to  the  raw  data. 

Participation:  Participation  is  VOLUNTARY.  No  adverse  action  will  be  taken  agamst  any 
member  who  does  not  participate  in  this  survey  or  who  does  not  complete  any  part  of  the  survey 
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Instructions 

This  questionnaire  is  part  of  a  leadership  development  program  managed  by  the  Air 
Force  Institute  of  Technology  (AFIT).  The  Leadership  Commitments  Inventory- Self 
provides  you  the  opportunity  to  assess  your  own  leadership  behaviors.  This  survey  will 
provide  information  that  will  allow  you  to  compare  your  perceptions  of  your  leadership 
behaviors  to  the  perceptions  of  others. 

We  have  developed  an  electronic  survey  to  reduce  material  costs  associated  with 
collecting  and  entering  data.  Several  steps  have  been  taken  to  protect  your  anonymity. 
First,  your  responses  will  be  sent  directly  to  the  AFIT  survey  control  point.  No  one  in 
your  organization  will  see  your  completed  survey.  Second,  you  are  the  only  person  who 
will  be  able  to  access  your  feedback. 

Please  contact  us  at  lci@afit.edu  if  you  have  any  questions  about  this  survey. 

We  thank  you  for  your  participation. 
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Please  read  each  statement  carefully  and  indicate  the  extent  to  which  you  engage  in  each 
of  the  behaviors.  Answer  in  terms  of  how  you  typically  act  with  and  on  behalf  of  your 
unit.  Use  the  blank  space  at  the  beginning  of  each  statement  to  record  the  number  of  your 
choice. 


1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in 
awhile 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

1.  I  communicate  to  others  what  I  am  passionate  about. 

2.  I  act  in  ways  that  are  consistent  with  my  stated  values. 

3.  I  clearly  explain  a  vision  of  the  team’s  future. 

4.  I  appeal  to  each  member’s  desire  to  contribute  to  the  success  of  the 
organization. 

5.  I  devote  time  to  discuss  innovation  and  change. 

6.  I  am  willing  to  experiment  with  new  ideas. 

7.  I  assign  tasks  that  require  team  members  to  cooperate  with  each  other. 

8.  I  allow  others  to  decide  the  best  way  to  get  their  jobs  done. 

9.  I  publicly  reward  individual  members  when  they  do  a  good  job. 

10.  I  make  sure  leaders  know  about  the  unit’s  successes. 

11.  I  allow  humor  to  break  through  during  tense  moments. 

12.  I  encourage  simple,  quick,  and  fun  activities  that  lift  spirits  at  work. 
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1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in 
awhile 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

13  I  share  my  personal  values  with  team  members. 

14.  I  lead  by  example. 

15.  I  create  vivid  images  that  help  convey  our  mission. 

16.  I  help  focus  the  team  on  a  common  purpose. 

17.  I  encourage  others  to  seek  out  better  ways  of  doing  things. 

18.  I  encourage  people  to  take  risks. 

19.  I  build  long-term  relationships  with  others  outside  of  the  unit. 

20.  I  give  others  important  work  to  do  on  critical  tasks. 

21.  I  reward  only  those  who  meet  or  exceed  challenging  standards. 

22.  I  show  appreciation  for  the  team’s  hard  work. 

23.  I  encourage  non-offensive  humor  as  a  way  to  make  the  workplace  more 
fun. 

24.  I  find  ways  to  offset  hardships  caused  by  work  with  some  fun  outcome  or 
activity. 
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1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in 
awhile 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

25.  I  clarify  to  others  what  leads  me  to  become  impatient. 

26.  I  make  decisions  that  are  consistent  with  my  stated  values. 

27.  I  portray  the  unit  as  having  an  important  impact  on  the  future. 

28.  I  help  team  members  relate  their  own  aspirations  with  the  unit’s 
mission. 

29.  I  look  for  ways  that  challenge  the  status  quo. 

30.  I  find  ways  to  turn  setbacks  into  learning  events. 

31.  I  encourage  the  open  exchange  of  information  and  ideas. 

32.  I  grant  team  members  the  appropriate  authority  to  do  their  work. 

33.  I  tailor  rewards  to  things  we  each  individually  value. 

34.  I  take  part  in  celebrating  team  accomplishments 

35.  I  am  not  afraid  to  laugh  at  myself. 

36.  I  take  advantage  of  lulls  in  the  schedule  for  relaxing  and  fun  activities. 
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1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in 
awhile 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

37.  I  share  with  others  what  keeps  me  awake  at  night. 

38.  I  set  a  personal  example  of  what  is  expected  of  unit  members. 

39.  I  provide  a  vision  that  helps  the  team  stay  energized,  focused,  and 
confident. 

40.  I  direct  my  team  member’s  attention  to  common  goals  that  can  be 
supported  by  all  members  of  the  group. 

41.  I  look  for  ways  to  improve  the  unit’s  effectiveness. 

42.  I  take  measured  risks  based  on  the  team’s  capabilities. 

43.  I  persuade  the  team  to  cooperate  with  others  in  order  to  build  strong 
partnerships. 

44.  I  make  sure  that  the  team  has  the  ability  to  make  good  judgments  on  its 
own. 

45.  I  take  note  of  high  performers. 

46.  I  take  time  out  to  publicly  recognize  the  unit’s  endeavors. 

47.  I  am  willing  to  laugh  and  have  fun  with  others. 

48.  I  am  willing  to  take  a  break  during  busy  periods  to  do  something  fun  as  a 
unit. 
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Appendix  H:  E-mail  template  for  LCI-Observer 


To:  Blank 

From:  user’s  e-mail  address  is  automatically  input  by  system 
Subject:  Request  for  your  feedback 
Body  of  e-mail: 

[INSERT  OBSERVERS’  NAMES], 

Please  help  me  get  an  honest  assessment  of  my  leadership  practices.  Click  the  link  below 
and  you  will  be  guided  through  the  survey  process.  You  will  be  prompted  to  provide  my 
identification  at  the  beginning  of  the  survey.  My  user  identifier  is  [generated  number] . 
You  will  also  be  asked  to  select  your  relationship  to  me.  Please  select  [INSERT  BOSS, 
PEER,  DIRECT  REPORT,  INDIRECT  REPORT,  TEAM  MEMBER], 

The  survey  should  take  about  10  minutes.  If  you  have  any  questions  concerning  the 
survey  process  please  contact  the  administrators  at  lci@afit.edu. 

<http://en.afit.edu/env/lci/lci  observer.cfm> 

The  developmental  feedback  survey  is  based  on  Kouzes  and  Posner's  book,  The 
Leadership  Challenge.  The  purpose  of  the  survey  is  to  obtain  information  regarding  your 
perceptions  of  my  leadership  behaviors.  The  survey  contains  items  measuring  12 
different  leadership  commitments.  Answer  every  question  based  on  the  extent  you  see 
me  typically  engaging  in  each  behavior.  All  of  the  questions  are  applicable.  If  you  don't 
think  a  statement  applies,  then  it  is  probably  because  I  don’t  do  this  type  of  behavior  very 
often. 

The  administrators  of  this  program  have  taken  several  steps  to  protect  your  anonymity 
and  ensure  that  my  feedback  is  only  accessible  by  me,  for  leadership  development.  I  will 
not  have  access  to  your  specific  assessment.  Your  responses  will  be  combined  with  the 
responses  from  the  other  peers,  direct  reports,  indirect  reports,  and  team  members  I  have 
chosen.  Further,  my  assessment  will  be  only  accessible  by  me.  No  one  in  my  leadership 
chain  will  ask  for  a  copy  of  my  developmental  feedback  report. 

Thank  you  for  your  participation. 

[SIGNATURE  BLOCK]  (User’s  name  is  automatically  input  by  system) 
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Appendix  I:  LCI  Observer  Assessment 


Air  Force  Institute  of  Technology 


Leadership  Commitments  Inventory 


Observer 


Privacy  Notice 

The  following  information  is  provided  as  required  by  the  Privacy  Act  of  1974 

Purpose;:  To  obtain  Information  regarding  employees'  perceptions  of  their  leaders',  peers', 
subordinates',  and  learn  members'  leadership  behaviors 

Routine  User  The  survey  results  will  be  used  to  provide  developmental  feedback  for  individual 
leaders  No  analysis  of  individual  responses  will  be  conducted  and  only  members  of  the  Air 
Force  Institute  of  Technology  research  team  will  be  permitted  access  to  the  raw  data. 

Participation;  Participation  is  VOLUNTARY  No  adverse  action  will  be  taken  agamst  any 
member  who  does  hot  participate  in  this  survey  or  who  does  not  complete  any  part  of  the  survey 


Please  input  code  provided  to  you  in  the  email  here.  (Ex.  10,  101,  1234....) 

*  Please  choose  the  option  that  best  states  your  relationship  to 

C  Boss 
C  Peer 
C  Direct  Report 
^  Indirect  Report 

c 


Team  Member 
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Instructions 

This  questionnaire  is  part  of  a  leadership  development  program  managed  by  the  Air 
Force  Institute  of  Technology  (AFIT).  The  Leadership  Commitments  Inventory  provides 
you  the  opportunity  to  give  a  leader,  peer,  subordinate,  or  team  member  specific  feedback 
on  his  or  her  leadership  behaviors.  Your  response  to  this  questionnaire  will  be  combined 
with  the  responses  of  the  other  members  of  your  unit.  The  AFIT  team  will  provide  this 
person  feedback  on  his  or  her  performance.  AFIT  will  also  provide  guidance  to  this 
leader  to  help  him  or  her  interpret  the  feedback,  develop  plans  to  act  on  the  feedback,  and 
share  the  action  plans  with  you. 

We  have  developed  an  electronic  survey  to  reduce  material  costs  associated  with 
collecting  and  entering  data.  Several  steps  have  been  taken  to  protect  your  anonymity 
and  ensure  that  this  person  cannot  identify  your  survey  responses.  First,  your  responses 
will  be  sent  directly  to  the  AFIT  survey  control  point.  No  one  in  your  organization  will 
see  your  completed  survey.  Second,  this  person  will  receive  an  assessment  of  his  or  her 
leadership  behaviors  based  on  the  combined  scores  of  all  the  people  that  participated.  In 
order  to  protect  your  privacy,  a  minimum  of  two  people  within  each  observer  category 
must  respond  for  this  person  to  receive  any  feedback  at  all.  Third,  no  one  other  than  this 
person  will  be  able  to  access  his  or  her  feedback.  This  protects  his  or  her  privacy. 

Please  contact  us  at  lci(a),  a  fit.edu  if  you  have  any  questions  about  this  survey.  We  thank 
you  for  your  participation. 
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Please  read  each  statement  carefully  and  indicate  the  extent  to  which  you  have  observed 
this  person  doing  each  of  the  behaviors.  Answer  in  terms  of  how  this  person  typically 
acts  with  you,  with  people  in  your  unit,  and  on  behalf  of  you  and  your  unit. 


1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in  a 
while 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

1.  Communicates  what  he  or  she  is  passionate  about. 

2.  Acts  in  ways  that  are  consistent  with  stated  values. 

3.  Clearly  explains  a  vision  of  the  team’s  future. 

4.  Appeals  to  each  member’s  desire  to  contribute  to  the  success  of  the  team. 

5.  Devotes  time  to  discuss  innovation  and  change. 

6.  Willing  to  experiment  with  new  ideas. 

7.  Assigns  tasks  that  require  team  members  to  cooperate  with  each  other. 

8.  Allows  team  members  to  decide  the  best  way  to  get  their  jobs  done. 

9.  Publicly  rewards  individual  members  when  they  do  a  good  job. 

10.  Makes  sure  leaders  know  about  the  team’s  successes. 

11.  Allows  humor  to  break  through  during  tense  moments. 

12.  Encourages  simple,  quick,  and  fun  activities  that  lift  spirits  at  work. 
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1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in  a 
while 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

13.  Shares  personal  values  with  team  members. 

14.  Leads  by  example. 

15.  Creates  vivid  images  that  help  convey  the  team’s  mission. 

16.  Helps  focus  the  team  on  a  common  purpose. 

17.  Encourages  others  to  seek  out  better  ways  of  doing  things. 

18.  Encourages  people  to  take  risks. 

19.  Builds  long-term  relationships  with  others  outside  of  the  team. 

20.  Gives  team  members  important  work  to  do  on  critical  tasks. 

21.  Rewards  only  those  who  meet  or  exceed  challenging  standards. 

22.  Shows  appreciation  for  the  team’s  hard  work. 

23.  Encourages  non-offensive  humor  as  a  way  to  make  the  workplace  more 
fun. 

24.  Finds  ways  to  offset  hardships  caused  by  work  with  some  fun  outcome  or 
activity. 


155 


1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in  a 
while 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

25.  Clarifies  to  others  what  leads  him  or  her  to  become  impatient. 

26.  Makes  decisions  that  are  consistent  with  his  or  her  stated  values. 

27.  Portrays  the  team  as  having  an  important  impact  on  the  future. 

28.  Helps  team  members  relate  their  own  aspirations  with  the  team’s 
mission. 

29.  Looks  for  ways  that  challenge  the  status  quo. 

30.  Finds  ways  to  turn  setbacks  into  learning  events. 

31.  Encourages  the  open  exchange  of  information  and  ideas. 

32.  Grants  team  members  the  appropriate  authority  to  do  their  work. 

33.  Tailors  rewards  to  things  each  team  member  individually  values. 

34.  Takes  part  in  celebrating  team  accomplishments. 

35.  Not  afraid  to  laugh  at  himself/herself. 

36.  Takes  advantage  of  lulls  in  the  schedule  for  relaxing  and  fun  activities. 
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1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in  a 
while 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

37.  Shares  with  team  members  what  keeps  him  or  her  awake  at  night. 

38.  Sets  a  personal  example  of  what  is  expected  of  team  members. 

39.  Provides  a  vision  that  helps  the  team  stay  energized,  focused,  and 
confident. 

40.  Directs  our  attention  to  common  goals  that  can  be  supported  by  all  team 
members. 

41.  Looks  for  ways  to  improve  the  team’s  effectiveness. 

42.  Takes  measured  risks  based  on  the  team’s  capabilities. 

43.  Persuades  team  to  cooperate  with  others  in  order  to  build  strong 
partnerships. 

44.  Makes  sure  team  members  have  the  ability  to  make  good  judgments  on 
their  own. 

45.  Takes  note  of  high  performers. 

46.  Takes  time  out  to  publicly  recognize  the  team’s  endeavors. 

47.  Willing  to  laugh  and  have  fun  with  others. 

48.  Willing  to  take  a  break  during  busy  periods  to  do  something  fun  as  a  team. 
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We  are  always  looking  for  ways  to  improve  the  developmental  feedback  process  and  our 
web  site.  Please  read  the  statements  carefully  and  indicate  the  extent  to  which  you  agree 
or  disagree  with  each  of  the  items. 


1  2 

3 

4 

5 

6 

7 

Strongly  Disagree 

Disagree 

Slightly 

Disagree 

Neither 
Agree  nor 
Disagree 

Slightly 

Agree 

Agree 

Strongly 

Agree 

49.  I  feel  I  have  worked  long  enough  with  this  person  to  adequately  assess 
his/her  leadership  behaviors. 

50.  I  believe  the  information  collected  by  this  survey  can  help  leaders  become 
more  effective. 

51.  The  survey  questions  were  easy  to  understand. 

52.  Learning  to  use  this  web  survey  was  easy  for  me. 

53.  I  found  it  easy  to  get  the  web  survey  to  do  what  I  wanted  it  to  do. 

54.  I  found  this  web  survey  easy  to  use. 

Please  feel  free  to  give  us  additional  comments  or  feedback. 
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Appendix  J:  E-mail  template  for  Stakeholder  Assessment 


To:  Blank 

From:  user’s  e-mail  address  is  automatically  input  by  system 
Subject:  Request  for  your  feedback 
Body  of  e-mail: 

[INSERT  STAKEHOLDERS’  NAMES], 

Please  help  me  get  an  honest  assessment  of  my  team’s  effectiveness.  Click  the  link 
below  and  you  will  be  guided  through  a  survey  that  measures  your  perceptions  of  our 
responsiveness,  professionalism  and  performance.  You  will  be  prompted  to  provide  my 
identification  at  the  beginning  of  the  survey.  My  user  identifier  is  [generated  number] . 

The  survey  should  take  about  10  minutes.  If  you  have  any  questions  concerning  the 
survey  process  please  contact  the  administrators  at  lci@afit.edu. 

http://en.afit.edu/env/lci/stake  1  .cfm 

The  administrators  of  this  program  have  taken  several  steps  to  protect  your  anonymity 
and  ensure  that  my  feedback  is  only  accessible  by  me,  for  my  teams’  development.  I  will 
not  have  access  to  your  specific  assessment.  Your  responses  will  be  combined  with  the 
responses  from  the  other  critical  stakeholders  I  have  chosen.  Further,  my  assessment  will 
be  only  accessible  by  me.  No  one  in  my  leadership  chain  will  ask  for  a  copy  of  my  unit’s 
report. 

Thank  you  for  your  participation. 

[SIGNATURE  BLOCK]  (User’s  name  is  automatically  input  by  system) 
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Appendix  K:  Stakeholder  Assessment 


This  questionnaire  is  part  of  a  leadership  development  program  for  leaders.  The 
items  provide  you  an  opportunity  to  give  the  leader  feedback  on  his  or  her  unit’s 
performance.  To  assure  your  anonymity,  your  response  to  this  questionnaire  will  be 
combined  with  the  responses  of  other  stakeholders  before  feedback  is  given  to  the  leader. 

Think  about  all  the  different  organizations  you  interact  with  on  a  regular  basis  and  use  the 
following  scale  to  rate  the  quality  of  your  interactions  with  this  unit  compared  to  the 
others. 


1  2  3  4  5  6  7 

One  of  the  Far  worse  Not  as  good  About  the  Better  than  As  good  as  Better  than 

worst  units  than  most  as  most  same  as  most  most  units  the  best  any  other 
units  units  units  units  unit 


1 .  Information  I  get  from  this  unit  is  free  from  error. 

2.  I  can  readily  make  sense  of  the  infonnation  I  get  from  this  unit. 

3.  Information  provided  by  this  unit  is  up  to  date. 

4.  The  information  I  get  from  this  unit  is  complete. 

5.  I  can  count  on  the  quality  of  information  I  get  from  this  unit  over  time. 

6.  The  products  and  services  from  this  unit  are  delivered  when  I  need  them. 

7.  The  unit  adapts  products  and  services  to  meet  my  unique  requirements. 

8.  This  unit’s  products  and  services  are  free  from  error. 

9.  I  can  count  on  the  quality  of  the  products  and  services  I  get  from  this  unit 
over  time. 

10.  The  products  and  services  I  get  from  this  unit  meet  my  needs. 

11.  I  find  that  members  of  this  unit  are  courteous. 

12.  Members  of  this  unit  are  knowledgeable  about  their  area  of  expertise. 

13.  Members  of  this  unit  conduct  themselves  in  a  professional  manner. 

14.  Members  of  this  unit  respond  to  my  requests  quickly. 

15.  The  members  of  this  unit  easily  adapt  to  unexpected  requirements. 

16.  Members  of  this  unit  listen  to  my  requests  before  acting. 

17.  Members  of  this  unit  seem  to  care  about  what  I  have  to  say. 
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We  would  also  like  to  convey  your  general  feelings  about  this  unit  and  its  products.  Read 
each  item  carefully  and  use  the  following  scale  to  indicate  the  extent  you  agree  with  each 
statement. 


1 

2 

3 

4 

5 

6 

7 

Strongly 

Disagree 

Slightly 

Neither 

Slightly 

Agree 

Strongly 

Disagree 

Disagree 

Agree  or 
Disagree 

Agree 

Agree 

1 .  Overall,  I  am  satisfied  with  the  relationships  I  have  the  members  of  this  unit. 

2.  Overall,  I  am  satisfied  with  the  products  and  services  I  get  from  this  unit. 

3.  I  get  high  quality  products  and  services  from  this  unit. 

4.  If  given  alternatives,  I  would  turn  to  this  unit  to  provide  the  products  and 
services  I  need. 


Please  identify  three  things  that  this  unit  does  exceptionally  well. 

1. _ 

2. _ 

3. _ 

Please  identify  three  areas  where  this  unit  could  improve. 

1. _ 

2. _ 

3. _ 


Please  include  any  other  comments  you  wish  to  share  with  the  leader. 


Thank  you  for  your  participation! 
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Appendix  L:  Understanding  the  Reports  page 

These  reports  summarize  your  responses  to  the  Leadership  Commitments 
Inventory  -Self  (a.k.a.  LCI-Self)  as  well  as  the  feedback  you  received  from  others.  The 
LCI  surveys  collect  information  using  48  examples  of  effective  leadership  behaviors. 
The  behaviors  are  grouped  to  form  six  leadership  practices  and  12  leadership 
commitments. 

The  menu  provides  choices  for  three  different  reports:  Summary,  Comparison, 
and  Stakeholder.  Choose  the  report  you  wish  to  view  by  clicking  the  appropriate  link. 
Below  are  explanations  of  the  reports  and  how  to  analyze  them.  If  you  have  questions 
about  the  reports  please  feel  free  to  contact  the  Air  Force  Institute  of  Technology 
Leadership  Commitments  Inventory  (LCI)  Research  Team  at  lci@afit.edu. 

Summary  Report 

The  summary  report  is  the  first  report  you’ll  want  to  view.  This  report  provides 
you  an  overall  picture  of  frequency  that  you  and  others  think  you  engage  in  actions 
consistent  with  the  12  leadership  commitments. 

The  “Summary  Feedback”  page  provides  information  at  the  practice  and 
commitment  levels.  The  “Your  Score”  column  provides  the  results  of  your  self- 
assessment.  The  other  categories  (i.e.,  Boss’s  Score,  Peers’  Scores,  etc.)  show  your 
respondents’  scores.  These  scores  are  based  on  the  same  7-point  scale  as  the  self- 
assessment. 


1 

2 

3 

4 

5 

6 

7 

Almost 

never 

Once  in  a 
while 

Occasionally 

Sometimes 

Usually 

Quite 

often 

Almost 

always 

To  protect  your  respondents’  anonymity,  all  scores  listed  under  these  categories 
(with  the  exception  of  the  Boss’s  category)  are  the  average  scores  of  your  respondents.  If 
there  is  no  information  in  a  particular  category  simply  click  on  that  header  to  hide  the 
column.  The  “Definitions”  link  on  this  page  will  take  you  to  the  practice/commitment 
definitions  for  reference,  if  necessary. 

You  can  also  get  information  for  specific  leadership  behavior  for  any  commitment 
by  clicking  on  the  practice  or  commitment  label.  Each  practice  and  commitment  links 
you  to  the  “Behavior  Feedback”  page  and  a  summary  of  behaviors  for  that  practice.  The 
behavior  summary  contains  scores  for  each  of  the  individual  behaviors,  which  are  written 
from  an  observer’s  perspective.  As  stated  above,  scores  listed  under  the  other  categories 
(with  the  exception  of  the  Boss’s  category)  are  the  average  scores  of  your  respondents. 
Each  practice  or  commitment  header  is  a  link  that  will  take  you  to  the  definitions  page  for 
reference,  if  necessary. 


162 


Comparison  Report 

Comparison  reports  allow  you  to  relate  your  perceptions  with  others.  These 
reports  contrast  your  score  with  the  score  of  your  boss,  or  with  the  average  score  of  your 
peers,  team  members,  direct  reports,  and  indirect  reports  for  each  of  the  12  leadership 
commitments.  We  derived  each  score  for  peers,  team  members,  and  direct/indirect 
reports  by  calculating  the  average  across  all  items  relating  to  that  commitment.  The 
responses  of  your  respondents  in  each  category  were  combined  in  order  to  protect 
anonymity.  The  table  presents  the  difference  between  your  self-assessment  and  your 
respondents  ratings.  The  commitments  are  then  ranked  based  on  this  difference  from 
lowest  to  highest.  A  negative  difference  implies  that  you  rated  yourself  higher  than  the 
average  rating  of  your  direct  reports.  The  bar  graph  is  a  pictorial  representation  of  this 
difference.  In  the  final  column  you  will  find  an  indication  of  the  variability  of  the 
responses.  "Their  Spread"  is  the  standard  deviation  of  the  responses.  A  large  value  in 
this  column  (e.g.,  greater  than  1.5)  indicated  that  the  people  providing  you  feedback 
tended  to  offer  very  different  responses  to  a  particular  item,  commitment,  or  practice. 

In  some  cases  you  may  find  that  your  own  perceptions  are  not  the  same  as  your 
respondents.  Pay  particular  attention  to  these  areas.  You  will  find  that  this  can  be  a 
good  place  to  start  as  you  work  to  develop  your  leadership  skills. 

Stakeholder  Report 

Many  feel  that  a  unit’s  perfonnance  is  directly  related  to  its  leadership.  The 
stakeholder  survey  is  designed  to  measure  the  respondents’  perceptions  of  unit 
performance.  In  this  case,  it  can  give  you  a  good  idea  of  the  strengths  and  weaknesses  of 
your  organization  and  provide  you  with  infonnation  you  can  act  upon  to  improve  your 
organization. 

This  report  presents  your  respondents’  scores  to  the  stakeholder  survey.  We 
derived  each  score  by  calculating  the  average  across  all  items  on  the  survey.  Items 
focusing  on  quality  of  information,  products,  services,  responsiveness,  and 
professionalism  are  based  on  the  7-point  scale  below. 


1 

2 

3 

4 

5 

6 

7 

One  of  the 
worst  units 

Far  worse 
than  most 
units 

Not  as  good  as 
most  units 

About  the 
same  as  most 
units 

Better  than 
most  units 

As  good  as 
the  best 
units 

Better  then 
any  other 
unit 

Items  focusing  on  general  feelings  about  the  unit  and  its  products  were  measured 
with  the  7-point  scale  below. 


1 

2 

3 

4 

5 

6 

7 

Strongly 

disagree 

Disagree 

Slightly 

disagree 

Neither 
agree  or 
disagree 

Slightly 

agree 

Agree 

Strongly 

agree 
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As  you’ll  see,  there  are  also  three  items  for  the  respondents  to  provide  written 
feedback.  These  items  allow  respondents  to  provide  comments  on  what  the  unit  does 
well  and  how  it  could  improve.  It  also  allows  respondents  to  provide  any  general 
comments  they  wish  to  share  with  the  leader.  All  the  comments  for  each  question  are 
listed  under  the  appropriate  category. 

We  hope  this  gives  you  a  good  idea  of  how  to  view  and  analyze  the  reports 
available  to  you.  If  you  have  any  questions  or  problems  during  the  reports  process, 
please  feel  free  to  contact  the  Air  Force  Institute  of  Technology  Leadership 
Commitments  Inventory  (LCI)  Research  Team  at  lci@afit.edu..  We  are  always  looking 
for  ways  to  improve  the  developmental  feedback  process  and  this  web  site.  If  you  have 
any  feedback  for  us,  please  contact  us.  Thank  you. 


Principles 


MODEL  THE  WAY 
INSPIRE  A  SHARED  VISION 
CHALLENGE  THE  PROCESS 
ENABLE  OTHERS  TO  ACT 
ENCOURAGE  THE  HEART 
ENJOY  THE  WORKPLACE 


Commitments 


MODEL 

Shares  personal  values 
Sets  the  example 
INSPIRE 
Creates  a  vision 

Attracts  others  to  a  common  purpose 
CHALLENGE 
Seeks  innovation 

Takes  risks  and  learns  from  mistakes 
ENABLE 

Promotes  cooperation 
Empowers  others 
ENCOURAGE 

Recognizes  individual  contributions 
Celebrates  team  accomplishments 
ENJOY 

Opens  door  for  humor 
Promotes  fun  activities 
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Direct  Reports' 
Score 
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Behavior  Feedback 


Practices,  Commitments  &  Behaviors 

Your 

Boss's 

Peers' 

Direct  Reports' 

Indirect  Reports' 

Team  Members' 

Score 

Score 

Score 

Score 

Score 

Score 

MODEL  THE  WAY 


Shares  personal  values 


1.  Communicates  what  he  or  she  is  passionate 
about. 

13.  Shares  personal  values  with  team  members. 


25.  Clarifies  to  others  what  leads  his  or  her  to 
become  impatient. 

37.  Shares  with  team  members  what  keep  him  or 
her  awake  at  night. 

Sets  the  example 


2.  Acts  in  ways  that  are  consistent  with  stated 
values. 

14.  Leads  by  example. 


26.  Makes  decisions  that  are  consistent  with  his  or 
her  stated  values. 

38.  Sets  a  personal  example  of  what  is  expected  of 
team  members. 


os 


Appendix  N :  Behavior  Summary  Report  Template 


Behavior  Feedback 


Practices,  Commitments  &  Behaviors 

Your 

Boss's 

Peers' 

Direct  Reports' 

Indirect  Reports' 

Team  Members' 

Score 

Score 

Score 

Score 

Score 

Score 

INSPIRE  A  SHARED  VISION 


Create  a  vision 


3.  Clearly  explains  a  vision  of  the  team’s  future. 

15.  Creates  vivid  images  that  help  convey  the 
team's  mission. 


27.  Portrays  the  unit  as  having  an  important  impact 
on  the  future. 

39.  Provides  a  vision  that  helps  the  team  stay 
energized,  focused,  and  confident. 

Attract  others  to  a  common  purpose 


4.  Appeals  to  each  member’s  desire  to  contribute 
to  the  success  of  the  team. 

16.  Helps  focus  the  team  on  a  common  purpose. 


28.  Helps  team  members  relate  their  own 
aspirations  with  the  team’s  mission. 

40.  Directs  our  attention  to  common  goals  that  can 
be  supported  by  all  team  members. 


Behavior  Feedback 


Practices,  Commitments  &  Behaviors 

Your 

Boss's 

Peers' 

Direct  Reports' 

Indirect  Reports' 

Team  Members' 

Score 

Score 

Score 

Score 

Score 

Score 

CHALLENGE  THE  PROCESS 


Seek  innovation 


5.  Devotes  time  to  discuss  innovation  and  change. 

17.  Encourages  others  to  seek  out  better  ways  of 
doing  things. 


29.  Looks  for  ways  that  challenge  the  status  quo. 

41.  Looks  for  ways  to  improve  the  team’s 
effectiveness. 

Take  risks  and  learn  from  mistakes 


6.  Willing  to  experiment  with  new  ideas. 
18.  Encourages  people  to  take  risks. 


30.  Finds  ways  to  turn  setbacks  into  learning 
events. 

42.  Takes  measured  risks  based  on  the  team’s 
capabilities. 


Behavior  Feedback 


Practices,  Commitments  &  Behaviors 

Your 

Boss's 

Peers' 

Direct  Reports' 

Indirect  Reports' 

Team  Members' 

Score 

Score 

Score 

Score 

Score 

Score 

ENABLE  OTHERS  TO  ACT 


Promote  Cooperation 


7.  Assigns  tasks  that  require  team  members  to 
cooperate  with  each  other. 

19.  Builds  long-term  relationships  with  others 
outside  of  the  team. 


3 1 .  Encourages  the  open  exchange  of  information 
and  ideas. 

43.  Persuades  team  to  cooperate  with  others  in 
order  to  build  strong  partnerships. 


Empower 


8.  Allows  team  members  to  decide  the  best  way  to 
get  their  jobs  done. 

20.  Gives  team  members  important  work  to  do  on 
critical  tasks. 


32.  Grants  team  members  the  appropriate  authority 
to  do  their  work. 

44.  Makes  sure  team  members  have  the  ability  to 
make  good  judgments  on  their  own. 


Behavior  Feedback 


Practices,  Commitments  &  Behaviors 

Your 

Boss's 

Peers' 

Direct  Reports' 

Indirect  Reports' 

Team  Members' 

Score 

Score 

Score 

Score 

Score 

Score 

ENCOURAGE  THE  HEART 


Recognize  individual  contributions 


9.  Publicly  rewards  individual  members  when 
they  do  a  good  job. 

21.  Rewards  only  those  who  meet  or  exceed 
challenging  standards. 


33.  Tailors  rewards  to  things  each  team  member 
individually  values. 

45.  Takes  note  of  high  performers. 

Celebrate  team  accomplishments 


10.  Makes  sure  leaders  know  about  the  team’s 
successes. 

22.  Shows  appreciation  for  the  team’s  hard  work. 


34.  Takes  part  in  celebrating  team 
accomplishments. 

46.  Takes  time  out  to  publicly  recognize  the  team’s 
endeavors. 


Behavior  Feedback 


Practices,  Commitments  &  Behaviors 

Your 

Boss's 

Peers' 

Direct  Reports' 

Indirect  Reports' 

Team  Members' 

Score 

Score 

Score 

Score 

Score 

Score 

ENJOY  THE  WORKPLACE 


Allow  humor  to  reduce  stress  and  boredom 


1 1 .  Allows  humor  to  break  through  during  tense 
moments. 

23.  Encourages  non-offensive  humor  as  a  way  to 
make  the  workplace  more  fun. 


35.  Not  afraid  to  laugh  at  himself/herself. 

47.  Willing  to  laugh  and  have  fun  with  others. 

Promote  fun  activities  to  relax  and  unwind 


12.  Encourages  simple,  quick,  and  fun  activities 
that  lift  spirits  at  work. 

24.  Finds  ways  to  offset  hardships  caused  by  work 
with  some  fun  outcome  or  activity. 


36.  Takes  advantage  of  lulls  in  the  schedule  for 
relaxing  and  fun  activities. 

48.  Willing  to  take  a  break  during  busy  periods  to 
do  something  fun  as  a  team. 


Comparison  Feedback 


Your 

Their 

The 

Their 

Score 

Score 

Difference 

Spread 

Sets  the  example  (Model) 

Attracts  others  to  vision  (Inspire) 

Celebrates  team  accomplishments  (Encourage) 
Creates  a  vision  (Inspire) 

Recognizes  individual  accomplishments  (Encourage) 
Takes  risks  and  learns  (Challenge) 

Shares  personal  values  (Model) 

Empowers  others  (Enable) 

Opens  door  for  humor  (Enjoy) 

Promotes  cooperation  (Enable) 

Promotes  fun  activities  (Enjoy) 

Seeks  innovation  (Challenge) 


Appendix  O:  Comparison  Report  Template 
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Appendix  P:  Stakeholder  Report  Template 


Stakeholder  feedback 

Quality  of  information,  products,  and  services 

Their 

scores 

Their 

spread 

Information  I  get  from  this  unit  is  free  from  error. 

I  can  readily  make  sense  of  the  information  I  get  from  this  unit. 

Information  provided  by  this  unit  is  up  to  date. 

The  information  I  get  form  this  unit  is  complete. 

I  can  count  on  the  quality  of  information  I  get  from  this  unit  over  time. 

I  can  count  on  the  quality  of  the  products  and  services  I  get  from  this  unit  over 
time. 

Responsiveness 

The  products  and  services  from  this  unit  are  delivered  when  I  need  them. 

The  unit  adapts  products  and  service  to  meet  my  unique  requirements. 

This  unit's  products  and  services  are  free  from  error. 

The  products  and  services  I  get  from  this  unit  meet  my  needs. 

The  members  of  this  unit  easily  adapt  to  unexpected  requirements. 

Members  of  this  unit  listen  to  my  requests  before  acting. 

Members  of  this  unit  seem  to  care  about  what  I  have  to  say. 

Professionalism 

I  find  that  members  of  this  unit  are  courteous. 

Members  of  their  unit  are  knowledgeable  about  their  area  of  expertise. 

Members  of  this  unit  conduct  themselves  in  a  professional  manner. 

Helps  focus  team  on  a  common  purpose. 

Feelings  about  unit  and  its  products 

Overall,  I  am  satisfied  with  the  relationships  I  have  with  the  members  of  this  unit. 

Overall,  I  am  satisfied  with  the  products  and  services  I  get  from  this  unit. 

I  get  high  quality  products  and  services  from  this  unit. 

If  given  alternatives,  I  would  turn  to  this  unit  to  provide  the  products  and  services 

I  need. 

Written  Comment  Items 

Please  identify  three  things  that  this  unit  does  exceptionally  well. 

Please  identify  three  areas  where  this  unit  could  improve. 

Please  include  any  other  comments  you  wish  to  share  with  the  leader. 
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Appendix  Q:  Guidance  on  Taking  Action  page 

Once  again,  thank  you  for  participating  in  the  Developmental  Feedback  Program. 
You  have  reached  the  final  step  in  the  process  and  it  is  time  to  take  action.  Here  is  a  little 
guidance  on  how  to  build  an  action  plan  based  on  the  feedback,  share  the  feedback  and 
action  plan  with  others,  and  follow  through  with  the  action  plan. 

Building  an  action  plan 

We  have  provided  an  action  plan  template  and  a  sample  action  plan  so  that  you 
can  build  an  action  plan  based  on  your  feedback  results.  You  will  be  able  to  use  the 
reports  to  determine  your  perceived  strengths  and  weaknesses.  After  this,  read  the 
suggested  actions  lists  and  look  at  the  suggested  readings.  These  will  give  you  ideas  on 
how  to  implement  changes  in  your  leadership  behaviors.  Remember  that  just  because 
you  scored  well  on  a  commitment  does  not  mean  you  can  ignore  it.  There  is  always  room 
for  improvement.  Print  the  action  plan  template  and  use  it  how  you  see  fit.  Or  we 
suggest  you  create  your  own  template  and  customize  an  action  plan  to  yourself. 

Sharing  the  feedback  and  action  plan  with  others 

Once  you  have  reviewed  your  feedback  and  developed  your  action  plan,  it  is  time 
to  share  the  results  with  others,  particularly  those  who  provided  you  with  the  feedback. 
You’ll  want  to  schedule  a  meeting  and  ensure  people  know  you  appreciate  the  honest 
feedback  they  gave  you.  At  the  meeting  it  is  important  to  explain  that  the  feedback  was 
anonymous.  Briefly  discuss  the  leadership  commitments  and  behaviors  associated  with 
those  commitments.  Show  them  the  reports  to  give  them  an  idea  of  your  scores  for  the 
commitments.  Share  your  perceived  strengths  and  weaknesses  and  how  you  plan  to 
improve  certain  areas.  Let  them  know  of  future  plans,  programs,  and  actions  that  may 
affect  them.  Discuss  the  comparison  report  and  the  gaps  between  your  scores  and  the 
scores  of  the  respondents.  Search  for  more  feedback  on  commitments  that  had  large 
gaps.  And  finally,  ask  for  more  feedback.  This  meeting  will  go  a  long  way  toward 
letting  your  people  know  you  respect  their  viewpoint. 

Follow  through  with  the  action  plan 

This  one  is  pretty  self-explanatory.  Finish  what  you  started.  Complete  all  action 
on  your  original  action  plan  and  then  create  a  new  one.  This  is  an  ongoing  process  that 
only  stops  when  you  do.  Six  months  or  so  from  now,  return  to  the  website  to  get  more 
feedback.  See  if  your  leadership  behaviors  have  improved! ! 

We  hope  this  gives  you  a  good  idea  of  how  to  take  action.  If  you  have  any 
questions  or  problems  during  the  process,  please  feel  free  to  contact  the  Air  Force 
Institute  of  Technology  Leadership  Commitments  Inventory  (LCI)  Research  Team  at 
lci@afit.edu.  We  are  always  looking  for  ways  to  improve  the  developmental  feedback 
process  and  this  web  site.  If  you  have  any  feedback  for  us,  please  contact  us.  Thank  you. 
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Appendix  R:  Suggested  Actions  and  Readings  pages 

MODEL  THE  WAY 

Leaders  who  Model  the  Way  demonstrate  high  standards  and  establish  clear 
expectations  for  individual  performance.  They  Share  Personal  Values  by  clarifying  and 
communicating  their  values  so  that  others  know  what  they  stand  for.  They  Set  the 
Example  by  demonstrating  daily  behaviors  that  teach  their  values  and  standards. 

Leaders  who  do  what  they  say,  spend  time  on  what  is  important,  and  lead  by  example 
appear  credible  to  others. 

The  following  are  a  few  ways  to  Model  the  Way: 

□  Know  what  is  important  to  you  and  spend  your  time  in  congruence  with  this 
importance 

□  Review  your  personal  values  frequently 

□  Set  high  performance  standards 

□  Set  personal  goals  and  work  toward  them  daily 

□  Accept  responsibility  and  strive  for  excellence 

□  Ensure  goals  are  specific,  measurable,  attainable,  and  challenging 

□  Ensure  organization  members  participate  in  goal  setting  process 

□  Focus  on  the  effort  of  improvement;  actual  improvement  will  follow 

□  Leam  to  both  reward  and  correct  people 

□  Focus  on  your  priorities;  remember  not  everything  can  be  the  most  important 

□  Keep  track  of  the  promises  and  commitments  you  make  and  carry  through  with  them 

□  Consider  team  members’  values  and  ethics 

□  Lead  from  the  front 

□  Do  something  dramatic  to  convey  your  values 

Here  are  some  readings  that  will  help  you  find  more  ways  to  Model  the  Way: 

Kouzes,  J.M.,  &  Posner,  B.Z.  (1993).  Credibility:  How  leaders  gain  and  lose  it,  why 
people  demand  it.  San  Francisco:  Jossey-Bass. 

Scwarzkopf,  H.N.,  with  Pietre,  P.  (1992).  It  doesn’t  take  a  hero.  New  York:  Bantam. 

Maister,  D.H.,  (2001).  Practice  what  you  preach:  What  managers  must  do  to  create  a 
high  achievement  culture.  New  York:  Free  Press. 

Maxwell,  J.C.  (1999).  The  21  indispensable  qualities  of  a  leader:  Becoming  the  person 
others  will  want  to  follow.  Nashville:  Thomas  Nelson,  Inc. 

Axelrod,  A.  (1999).  Patton  on  leadership:  Strategic  Lessons  for  corporate  warfare. 
Paramus,  NJ:  Prentice  Hall. 

Blanchard,  K.,  Robinson,  D.,  &  Robinson,  J.  (2002).  Zap  the  gaps!  Target  higher 
performance  and  achieve  it!  New  York:  HarperCollins. 
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INSPIRE  A  SHARED  VISION 

Leaders  who  Inspire  a  Shared  Vision  convey  a  vivid  image  of  the  future  and 
develop  a  general  understanding  of  that  vision  among  the  organization’s  members.  These 
leaders  Create  a  Vision  that  conveys  a  vivid  image  of  the  organization’s  future. 

Likewise,  they  Attract  Others  to  a  Common  Purpose  by  showing  and  communicating 
how  aspirations  are  mutually  beneficial  to  work  group  members  and  the  organization. 


The  following  are  a  few  ways  to  Inspire  a  Shared  Vision: 

□  Write  down  your  vision 

□  Help  employees  develop  strong  ties  with  each  other 

□  Mark  milestones  publicly 

□  Track  progress  and  share  data  with  everyone 

□  Use  new  technologies  to  help  people  see  your  vision 

□  Tell  vivid  stories  over  and  over  about  the  new  organizational  vision 

□  Create  a  moral  compass  to  bring  integrity  and  fuel  to  the  vision 

□  Create  an  intuitive  compass  to  fire  up  those  committed  to  the  vision 

□  Create  a  historical  compass  and  build  vision  on  the  past  positives 

□  Create  a  vision  that  is  a  connection  between  the  past,  present,  and  future 

□  Create  a  directional  compass  to  provide  direction  for  the  team 

□  Create  a  strategic  compass  to  bring  process  to  the  vision 

□  Create  a  visionary  compass  to  challenge  your  team  to  reach  its  potential 

□  Do  not  get  lost.  Keep  everyone  aligned  to  the  vision  to  maintain  purpose 

□  Align  with  a  vision  and  keep  your  team  focused,  energized,  and  confident 


Here  are  some  readings  that  will  help  you  find  more  ways  to  Inspire  a  Shared 
Vision: 

Blanchard,  K.  (1998).  Gung  ho!  Turn  on  the  people  in  any  organization.  New  York: 
William  and  Morrow  Co. 

Blanchard,  K.  (2002).  Zap  the  Gaps.  New  York:  Harper  Collins  Publisher. 

Hiam,  A.  (2002).  Making  Horses  Drink.  Canada:  Entrepreneur  Press. 

Kotter,  J.P.  (2002).  The  Heart  of  Change.  Boston:  Harvard  Business  School  Press. 

Maxwell,  J.C.  (1999).  The  21  Indispensable  Qualities  of  a  leader.  Nashville:  Thomas 
Nelson  Publishers. 

Maxwell,  J.C.  (2001).  The  17  Indisputable  laws  of  teamwork.  Nashville:  Thomas  Nelson 
Publishers. 
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CHALLENGE  THE  PROCESS 

Leaders  who  Challenge  the  Process  encourage  their  people  to  search  for 
opportunities  to  change  the  status  quo,  experiment,  take  risks,  and  then  leam  from 
mistakes.  These  leaders  Seek  Innovation  by  searching  for  and  encouraging  others  to 
search  for  opportunities  to  improve  the  efficiency  and  effectiveness  of  the  organization. 
Additionally,  they  Take  Risks  and  Learn  From  Mistakes  by  creating  opportunities  for 
team  members  to  experiment  with  new  ideas  in  order  to  gain  critical  knowledge  about  the 
best  ways  to  add  value  for  the  customer. 


The  following  are  a  few  ways  to  Challenge  the  Process: 

□  Ask  “what  if’  questions  all  the  time 

□  Keep  a  persistent,  open  mind 

□  Set  up  a  suggestion  system 

□  Have  an  idea  of  the  month 

□  Respect  ideas  to  make  improvement  contagious 

□  Establish  a  “creative  comer” 

□  Understand  and  manage  creative  roles 

□  Acknowledge  the  value  of  stepping  stone  ideas 

□  Innovate  in  pairs  to  maximize  communication 

□  Seek  ways  to  improve  or  replace  everything 

□  Ask  dumb  questions  to  get  people  thinking 

□  Keep  a  public  record  of  creative  suggestions 

□  Give  strange  ideas  serious  consideration 

□  Change  for  changes  sake 

□  Change  your  mindset 


Here  are  some  readings  that  will  help  you  find  more  ways  to  Challenge  the  Process: 

Blanchard,  K.  (1998).  Gung  ho!  Turn  on  the  people  in  any  organization.  New  York: 
William  and  Morrow  Co. 

Blanchard,  K.  (2002).  Zap  the  Gaps!  Target  higher  performance  and  achieve  it!  New 
York:  Harper  Collins  Publisher. 

Hiam,  A.  (2002).  Making  Horses  Drink.  Canada:  Entrepreneur  Press. 

Kotter,  J.P.  (2002).  The  Heart  of  Change.  Boston:  Harvard  Business  School  Press. 

Maxwell,  J.C.  (1999).  The  21  Indispensable  Qualities  of  a  leader.  Nashville:  Thomas 
Nelson  Publishers. 

Yukl,  G.  (2002).  Leadership  in  Organizations.  New  Jersey:  Prentice  Hall. 
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ENABLE  OTHERS  TO  ACT 

Leaders  who  Enable  Others  to  Act  foster  teamwork  among  the  organization’s 
members,  create  an  environment  of  mutual  respect  and  trust,  and  provide  members  with 
the  necessary  support  to  perform  at  their  best.  They  Promote  Cooperation  by 
encouraging  the  open  exchange  of  information  and  ideas  among  work  group  members. 
Likewise,  they  Empower  organization  members  by  providing  them  with  the  necessary 
knowledge,  infonnation,  and  authority  to  take  control  of  their  jobs. 


The  following  are  a  few  ways  to  Enable  Others  to  Act: 

□  Treat  organization  members  as  invaluable  members  of  the  team 

□  If  you  believe  it,  they  will  believe  it  and  will  act  in  that  manner 

□  Assign  critical  tasks  to  others 

□  Give  support  as  needed  to  help  them  be  successful 

□  Establish  a  forum  for  free  and  open  exchange  of  ideas 

□  Listen  to  the  ideas 

□  Assign  tasks  to  individuals  who  normally  do  not  perform  them 

□  Make  it  known  that  training  is  important  and  critical  to  the  organization’s  success 

□  Share  information  about  important  issues  going  on  in  the  organization 

□  Give  people  substantial  authority  to  make  decisions  on  their  own 

□  Give  organization  members  the  opportunity  to  be  in  charge 

□  Include  coworkers  outside  your  organization  in  planning  meetings  and  fun  events  too 

□  Encourage  team  member  self-improvement  efforts — on  duty  and  off  duty 

□  Teach  your  people  how  to  do  your  job 

□  Let  team  members  implement  improvement  ideas  themselves 


Here  are  some  readings  that  will  help  you  find  more  ways  to  Enable  Others  to  Act: 

Pfeffer,  J.  (1994).  Competitive  Advantage  through  people:  Unleashing  the  power  of  the 
work  force.  Boston:  Harvard  Business  School  Press. 

Lawler,  E.E.,  III.  (1992).  The  ultimate  advantage:  Creating  the  high-involvement 
organization.  San  Francisco:  Jossey-Bass. 

Maxwell,  J.C.  (1999).  The  21  indispensable  qualities  of  a  leader:  Becoming  the  person 
others  will  want  to  follow.  Nashville:  Thomas  Nelson,  Inc. 

Axelrod,  A.  (1999).  Patton  on  leadership:  Strategic  Lessons  for  corporate  warfare. 
Paramus,  NJ:  Prentice  Hall. 

Hudson,  F.M.  (1999).  The  handbook  of  coaching:  A  comprehensive  resource  guide  for 
managers,  executives,  consultants,  and  human  resource  professionals.  New 
York:  Jossey-Bass. 
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ENCOURAGE  THE  HEART 

Leaders  who  Encourage  the  Heart  set  high  expectations,  recognize  individuals  for 
their  progress  and  contributions,  provide  rewards  for  exceptional  perfonnance,  and 
celebrate  the  accomplishments  of  the  work  group.  They  Recognize  Individual 
Contributions  by  rewarding  individual  progress  and  contributions  that  meet  high 
standards  of  perfonnance.  They  also  Celebrate  Team  Accomplishments,  personally 
highlighting  and  recognizing  the  work  group’s  attainment  of  key  objectives  and  goals. 

The  following  are  a  few  ways  to  Encourage  the  Heart: 

□  Set  high  expectations  and  reward  people  when  they  meet  or  exceed  those 
expectations 

□  Tailor  rewards  to  recognize  specific  accomplishments  and  contributions  of  the 
individual 

□  Make  rewards  meaningful 

□  Recognize  progress,  not  just  results 

□  Recognize  success,  no  matter  how  small  the  accomplishment 

□  Say  thank  you  often 

□  Provide  frequent  feedback  and  recognition 

□  Get  out  from  behind  the  desk  and  find  people  doing  good  things 

□  Let  people  know  their  unique  characteristics  and  how  important  those  are  to  the  unit 

□  Make  people  tell  you  about  their  progress  and  accomplishments 

□  Make  individual  and  unit  recognition  a  public  event 

□  Celebrate  events  that  are  important  to  the  unit 

□  Schedule  events  to  show  appreciation  for  the  unit’s  hard  work 

□  Highlight  the  person’s  or  unit’s  strengths  when  they  face  challenges  or  difficulties 

□  Let  your  people  know  you  care 

Here  are  some  readings  that  will  help  you  find  more  ways  to  Encourage  the  Heart: 


Blanchard,  K.H.  &  Bowles,  S.  (1997).  GungHo!  Turn  on  the  people  in  any 
organization.  New  York:  William-Morrow  &  Co. 

Blanchard,  K.,  Lacinak,  T.,  Tompkins,  C.,  Ballard,  J.,  &  Blanchard,  K.  (2002).  Whale 
Done!  The  power  of  positive  relationships.  Free  Press. 

Hiam,  A.  (2002).  Chapter  8.  Encouragement.  Making  Horses  Drink.  Canada: 
Entrepreneur  Press. 

Kouzes,  J.M.  &  Posner,  B.Z.  (1998).  Encouraging  the  Heart:  A  leader’s  guide  to 
rewarding  and  recognizing  others.  San  Francisco:  Jossey-Bass. 

Nelson,  B.  (1994).  1001  Ways  to  Reward  Employees.  New  York:  Workman  Publishing. 
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ENJOY  THE  WORKPLACE 

Leaders  who  Enjoy  the  Workplace  create  a  playfully  productive  atmosphere  at  work 
by  encouraging  humor  and  promoting  fun  activities.  These  leaders  Allow  Humor  to 
Reduce  Stress  and  Boredom  by  encouraging  humor  to  break  tension  and  create  an 
enjoyable  workplace.  Likewise,  they  Promote  Fun  Activities  to  Relax  and  Unwind 
and  encourage  creative  and  fun  activities  to  increase  morale  and  job  satisfaction. 


The  following  are  a  few  ways  to  Enjoy  the  Workplace: 

□  Be  willing  to  laugh  and  have  fun  with  others 

□  Be  the  catalyst  for  fun  and  humor 

□  Take  work  seriously,  but  don’t  take  yourself  too  seriously 

□  Don’t  be  afraid  to  laugh  at  yourself 

□  Start  a  “joke-of-the-day”  tradition  (keep  ‘em  clean) 

□  Encourage  a  friendly  work  environment 

□  Find  time  for  fun  events  during  busy  times  (i.e.,  off-sites) 

□  Take  part  in  unit  social  activities 

□  Choose  informal  or  relaxing  settings  to  hold  stressful  meetings 

□  Break  the  ice  during  tense  moments 

□  Use  lulls  in  the  schedule  as  an  excuse  to  have  a  unit  picnic  or  party 

□  The  way  to  a  person’s  heart  goes  through  their  stomach  (take  ‘em  to  lunch) 

□  Create  a  spirit  award 

□  Use  quick,  simple,  and  fun  activities  to  lift  spirits  at  work 

□  And  finally,  don’t  forget  to  smile! 


Here  are  some  readings  that  will  help  you  find  more  ways  to  Enjoy  the  Workplace: 

Freiburg,  K.L.  &  Freiburg,  J.A.  (1996).  Nuts!  Southwest  Airlines’  crazy  recipe  for 
business  and  personal  success.  New  York:  Broadway  Books. 

Hemsath,  D.,  Yerkes,  L.,  &  McQuillen,  D.  (1997).  301  ways  to  have  fun  at  work. 
Williston,  VT :  Berrett-Koehler. 

Lundin,  S.C.,  Paul  H.,  &  Christensen  J.  (2000).  Fish!  A  remarkable  way  to  boost  morale 
and  improve  results.  New  York:  Hyperion. 

Lundin,  S.C.,  Paul  H.,  Christensen  J.,  &  Strand  P.  (2002).  Fish!  Tales:  Real  life  stories 
to  help  you  transform  you  workplace  and  your  life.  New  York:  Hyperion. 

Yerkes,  L.  (2001).  Fun  Works:  Creating  places  where  people  love  to  work.  Williston, 
VT:  Berrett-Koehler. 
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Appendix  S:  Sample  Action  Plan 

TAKING  ACTION 

My  Strengths  My  Weaknesses 

1 .  Set  the  example  (Model)  1 .  Recognize  individual  contributions 

2.  Promote  cooperation  (Enable)  (Encourage) 

3.  Seek  innovation  (Challenge)  2.  Create  a  vision  (Inspire) 

3 .  Allow  humor  (Enj  oy) 


Model 


My  Action  Plan 


Enable 


Completed  Due  date  Completed  Due  date 

□  Set  up  a  suggestion  system  22  Nov  □  Give  next  big  project  to  Jane  ASAP 

□  Seek  improvement  opp.  ongoing  □  Discuss  training  with  sections  3 1  Oct 


Inspire 

Completed 

Due  date 

□ 

Unit  vision  statement 

27  Sep 

□ 

Post  approved  vision 

4  Oct 

□ 

Develop  unit  milestones 

1  Nov 

Encourage 

Completed  Due  date 

□  Create  monthly  award  program  Christmas 

□  Celebrate  unit  10-yr  anniv.  18  Sep 

□  Check  project  status  and  recognize 

progress  This  week 


Challenge 

Completed  Due  date 

□  Talk  core  values  at  meeting  next  week 

□  Write  down  personal  goals  end  of  week 


Enjoy 

Completed  Due  date 

□  Start  “joke  of  the  day”  Monday 

□  Take  action  team  to  lunch  2  weeks 

□  Schedule  unit  picnic  end  of  mth 
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Appendix  T :  Action  Plan  Template 

TAKING  ACTION 

My  Strengths 

1. 

2. 

3. 

4. 


My  Weaknesses 

1. 

2. 

3. 

4. 


My  Action  Plan 


Model 


Enable 


Completed 

Due  date 

Completed 

□ 

□ 

□ 

□ 

□ 

□ 

Due  date 


Inspire 

Completed  Due  date 

□ 

□ 

□ 

Challenge 

Completed  Due  date 

□ 

□ 


Encourage 

Completed  Due  date 

□ 

□ 

□ 

Enjoy 

Completed  Due  date 

□ 

□ 


□ 
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